Download as
Browse Code »

Commit 5588ee4556b73c0b72ae04c3bfc235fbd5edece1

Authored by Paulo Meireles
1 parent ca3c632b
Exists in master and in 3 other branches oss_2018, oss_2018_siqueira, rework_case_study

[opensym] Trying 10 pages

Showing 12 changed files with 508 additions and 529 deletions   Show diff stats
opensym2017/content/01-introduction.tex
  Diff comments   View file @5588ee4
... ... @@ -3,14 +3,14 @@
3 3  
4 4 The Brazilian Government released in the year 2000 the Eletronic Government
5 5 program (eGov) aiming at democratizing information access and improving the
6   -public provision quality of service and information. In 2003, the Federal Government created a
  6 +public provision quality of service and information. In 2003, the Federal
  7 +Government created a
7 8 committee\footnote{\url{http://www.softwarelivre.gov.br/documentos-oficiais/
8 9 DecretoComite}} for implementation of Free/Libre/Open Source Software
9 10 (FLOSS\footnote{In this work, the acronym ``FLOSS'' is used as a representative
10 11 for ``Free Software'', ``Open Source Software'' (OSS), and``Free/Open Source
11   -Software'' (FOSS).}) and thereafter a
12   -circular-letter was sent to all Ministries in which the recommendation to adopt FLOSS
13   -became a public
  12 +Software'' (FOSS).}) and thereafter a circular-letter was sent to all
  13 +Ministries in which the recommendation to adopt FLOSS became a public
14 14 policy\footnote{\url{http://www.softwarelivre.gov.br/documentos-oficiais/circulardoministro}}.
15 15 In 2007, the Brazilian Public Software Portal (\textit{Portal do Software
16 16 Público Brasileiro}, in Portuguese) was released with the goal of sharing FLOSS
... ... @@ -82,9 +82,11 @@ can help other projects to overcome similar software engineering challenges in
82 82 the future, as well as to illustrate how universities can improve the
83 83 real-world experience of their students.
84 84  
85   -The remainder of this work is organized as follows. Section \ref{sec:spb}
86   -discusses the concepts of Brazilian Public Software and Free Software. Section
87   -\ref{sec:related} enumerates a number of related projects from other countries.
  85 +\begin{comment}
  86 +%FALTA ESPAÇO
  87 +The remainder of this work is organized as follows. Section \ref{sec:spb}
  88 +discusses the concepts of Brazilian Public Software and Free Software. Section
  89 +\ref{sec:related} enumerates some related projects from other countries.
88 90 Section \ref{sec:researchdesign} presents the open questions these guided this
89 91 paper. Section \ref{sec:requirements} reports how the Brazilian Government
90 92 stakeholders collected the theoretical requirements as well as how we define
... ... @@ -92,12 +94,9 @@ the technological requirements to release an initial version. Section
92 94 \ref{sec:architecture} shares our decisions about the systems that together
93 95 provided a wide subset of the requirements and our strategy to integrate them.
94 96 Section \ref{sec:features} describes the main features of the new SPB Portal.
95   -Section \ref{sec:ux} reports the user experience evolution during the
96   -integration of the selected FLOSS tools. Section \ref{sec:process} discusses
97   -our strategies to support the different organizational cultures and to involve
98   -undergraduate students as protagonists of the development process. Section
99   -\ref{sec:contributions} summarizes the contributions to the FLOSS upstream
100   -communities we interacted with. Finally, Sections \ref{sec:conclusion}
101   -concludes the paper highlighting its main contributions, sharing our lessons
102   -learned, and pointing paths to future works.
103   -
  97 +Section \ref{sec:process} discusses our strategies to support the different
  98 +organizational cultures and to involve undergraduate students as protagonists
  99 +of the development process. Finally, Sections \ref{sec:conclusion} concludes
  100 +the paper highlighting its main contributions, sharing our lessons learned, and
  101 +pointing paths to future works.
  102 +\end{comment}
... ...
opensym2017/content/03-relatedworks.tex
  Diff comments   View file @5588ee4
... ... @@ -11,10 +11,8 @@ making it easy for users and developers to obtain information and to access thei
11 11 source code repositories at GitHub. However, it does not provide social networking
12 12 and CMS features, as well as, other communication resources.
13 13  
14   -Additionally, there are two initiatives in Europe:
15   -OSOR\footnote{\url{https://joinup.ec.europa.eu/community/osor}} and
16   -OW2\footnote{\url{http://ow2.org}}. The Open Source Observatory (OSOR) is a
17   -community hosted in the JoinUp platform powered by the European Commission.
  14 +The European Commission supports the Open Source Observatory\footnote{\url{https://joinup.ec.europa.eu/community/osor}} (OSOR) that is a
  15 +community hosted in the JoinUp platform.
18 16 OSOR intends to exchange information, experiences, and best practices around the
19 17 use of FLOSS in the public administration. It supports the discovering of FLOSS projects made
20 18 available by public agencies, providing information about these projects, such
... ... @@ -23,13 +21,6 @@ It also offers discussion forums and community mailing lists. But it
23 21 does not have an integrated source code repository manager, so for each
24 22 project there is a link to its own external repository.
25 23 %
26   -OW2 is an organization that promotes the development, deployment and management
27   -of FLOSS middleware, generic
28   -business applications, and cloud computing platforms, besides fostering a community and
29   -business ecosystem around such projects.
30   -
31   -\leo{E aí, o que a OW2 tem a ver com o SPB?}
32   -
33 24 Moreover, in 2007 the European Comission published a study examining the impact
34 25 that the development and distribution of FLOSS by public agencies have on eGovernment
35 26 services and the economy~\cite{ghosh2004}. And, from 2007 to 2011, there
... ... @@ -61,6 +52,3 @@ avoiding the usage of private platforms, especially the ones provided by foreign
61 52 companies. Therefore, we needed to develop our own solution to cover all the
62 53 requirements, producing a complete governmental integrated platform for
63 54 collaborative software development.
64   -
65   -\leo{Me parece que o decreto fala sobre sistemas de comunicação. Acho que o foco era e-mail. Não me parece que o SPB tenha a ver com isso. A menos que se considere algo pelos fóruns... para refletir.}
66   -
... ...
opensym2017/content/06-architecture.tex
  Diff comments   View file @5588ee4
... ... @@ -104,7 +104,7 @@ stores the metrics evolution history, presents results in a friendly
104 104 way, as well as, allows users to customize the given interpretation
105 105 accordingly to their own context.
106 106  
107   -\subsection{System unification}
  107 +\subsection{System unification and User eXperience evolution}
108 108  
109 109 \begin{figure}[hbt]
110 110 \centering
... ... @@ -123,7 +123,33 @@ functionality of each application, a search o n the SPB portal might
123 123 return content from any of the applications, be it web pages, mailing
124 124 list posts, or source code.
125 125  
126   -% Falar do devops
  126 +The integration of collaborative environments goes beyond functional aspects.
  127 +Offering the population an unified experience across these environments has
  128 +been the key to encourage the use of the platform as it reduces the perception
  129 +of complexity. Thus, the SPB Portal information architecture was redesigned
  130 +to provide a transparent navigation and to reach users with different profiles.
  131 +A process of harmonization has been employed on the interaction models of each
  132 +tool to reduce the learning curve. At the same time, a new visual style was
  133 +created to unify the navigation experience and to comply with the guidelines of
  134 +the digital communication identity standard established by the Federal
  135 +Government.
  136 +
  137 +With the increase in system features and the addition of new tools, the
  138 +visual style has steadily evolved to keep the navigation unified. Moreover,
  139 +tools from different backgrounds, which in many cases provide similar
  140 +functionality, prompted the development of an unified interface. Some
  141 +features, such as search and user profile editing were eliminated from
  142 +the individual applications, and implemented centrally to ensure a
  143 +consistent look and feel.
  144 +
  145 +Another challenge was responsive web design. The integrated applications
  146 +had varying degrees of support for responsiveness, and the common
  147 +interface had to adapt for each individual scenario. In particular
  148 +Noosfero did not yet have a responsive design; we engaged in its
  149 +development and contributed towards that goal.
  150 +
  151 +\begin{comment}
  152 +%FALTA DE ESPAÇO
127 153 \subsection{Deploy}
128 154  
129 155 The SPB platform was deployed in 7 virtual machines with different functions,
... ... @@ -161,3 +187,4 @@ static analysis tools on source code stored in repository and provide this data
161 187 to Prezento. A social network and CMS (Content Manager System) is provided by
162 188 Noosfero in \textit{social}, and the databases of all tools with a cache
163 189 service are in \textit{database}.
  190 +\end{comment}
... ...
opensym2017/content/07-features.tex
  Diff comments   View file @5588ee4
... ... @@ -74,3 +74,4 @@ Thus, the SPB became a platform to stimulate the openness of the source code;
74 74 dialogue between users and the development team; and also
75 75 maintenance and evolution of the software, which will provide more
76 76 transparency in government investments.
  77 +
... ...
opensym2017/content/08-process.tex 0 → 100644
  Diff comments   View file @5588ee4
... ... @@ -0,0 +1,181 @@
  1 +\section{Development Organization and Process}
  2 +\label{sec:process}
  3 +
  4 +The SPB team was composed of a variety of professionals with different levels
  5 +and skills, where most of them were undergraduate students with major in
  6 +software engineering (from 4th semester or upper). Since the students could
  7 +not dedicate many hours per week to the project, they always had the
  8 +flexibility to negotiate their work schedule during the semester in
  9 +order to not harm their classes and coursework. Their daily work routine
  10 +in the project included programming and DevOps tasks.
  11 +
  12 +The development of SPB project required a vast experience and background that
  13 +usually undergraduate students do not have yet. For this reason, a few senior
  14 +developers have joined to the project to help with the more difficult issues
  15 +and to transfer knowledge to the students. Their main task was to provide
  16 +solutions for complex problems, in other words, they worked as developers. As
  17 +these professionals are very skillful and the project could not fund a full
  18 +time work for them, some of them worked partially on the project. In addition,
  19 +they lived in a different states spread around Brazil which led much of the
  20 +communication to be online.
  21 +
  22 +In short, our work process was based on open and collaborative software
  23 +development practices. The development process was defined based on the
  24 +adaptation of different agile and FLOSS communities practices, highlighting the
  25 +high degree of automation resulting from DevOps practices. Thus, the work
  26 +process was executed in a cadenced and continuous way.
  27 +
  28 +Finally, the last group of actors of this project was composed of employees
  29 +formally working for the Brazilian government, in the Ministry of Planning,
  30 +Development, and Management (MP is the Brazilian acronym). All the project
  31 +decisions, validations, and scope definitions were made by them. In this way we
  32 +developed software product incrementally, with releases aligned to business
  33 +strategic objectives. As one can see, the project had many different
  34 +sort of stakeholders that had to be organized and synchronized.
  35 +
  36 +\subsection{Team organization}
  37 +
  38 +Approximately 70\% of the development teams were composed of software
  39 +engineering undergraduate students from UnB and they worked physically
  40 +in the same laboratory. Each student had their own schedule based on
  41 +their classes, what complicates the implementation of pair programming.
  42 +The senior developers tried to synchronize their schedule with the
  43 +students on their sub-team. To cope with this environment, we had a few
  44 +basic rules which guided the project organization:
  45 +
  46 +\begin{enumerate}
  47 + \item Classes have high priority for undergraduate students;
  48 + \item Work in pairs whenever possible (locally or remotely);
  49 + \item There must be one morning or afternoon per week when
  50 + \emph{everyone} should be together physically in the laboratory
  51 + (except of course the remote team members);
  52 + \item Every 3 to 4 months, the senior developers would fly in and work
  53 + alongside the students for a few days.
  54 +\end{enumerate}
  55 +
  56 +With the aforementioned rules we divided all the project into four
  57 +different teams: Colab, Noosfero, Design, and DevOps. Each team had one
  58 +coach responsible for reducing the communication problem with the other
  59 +teams and help the members to organize themselves in the best way for
  60 +everyone (always respecting their work time). The coach was one of the
  61 +students working in some of the teams with the extra duty of registering
  62 +the current tasks developed in the sprint and with the responsibility to
  63 +talk with other teams. One important thing to notice is the mutability
  64 +of the team and the coach, during the project many students changed
  65 +between the teams to try different areas.
  66 +
  67 +One characteristic of the teams was the presence of (at least) one
  68 +senior per team. This was essential, because hard decisions and complex
  69 +problems were usually referred to them. This kept having to take
  70 +complicated technical decisions from the coach tole, what encouraged
  71 +students to be coaches. Lastly, the senior developers worked directly
  72 +with the students, and this was important to give the undergraduate the
  73 +opportunity to interact with a savvy professional in his area and to
  74 +keep the knowledge flowing in the project.
  75 +
  76 +Finally, we had to add two last elements of the team organization, that
  77 +was essential for the project harmony: the meta-coach and professors.
  78 +The former was a software engineer recently graduated and which wanted
  79 +to keep working on the project, the latter were professors that
  80 +orchestrated all the interactions between all members of the project.
  81 +The meta-coach usually worked in one specific team and had the extra
  82 +task of knowing the current status of all teams. Professors and
  83 +meta-coaches worked together to reduce the communication problem between
  84 +all the teams. Lastly, all the paperwork tasks, such as reporting on the
  85 +project progress to the Ministry, was handled by the professors.
  86 +
  87 +\subsection{Communication and management}
  88 +
  89 +Our team had many people working together, and most of the seniors worked in a
  90 +different city remotely. Also, we tried to keep our work completely clear to
  91 +the Brazilian government and citizens interested in following the project. To
  92 +handle those cases, we used a set of tools to communication and other to manage
  93 +the project.
  94 +
  95 +For communication between members in different places, we used: video
  96 +conferencing with shared terminal tools, IRC, and mailing lists. For example,
  97 +when one student had to work in pair with a senior, normally, they used video
  98 +conferencing tool for talking and shared a terminal session (both typing and
  99 +seeing the screen in real time). For questions and fast discussion, we used
  100 +IRC. For general notification, we used the mailing lists.
  101 +
  102 +For managing the project we used the SPB Portal itself; first to validate it by
  103 +ourselves, and also because it had all the required tools. We basically created
  104 +one wiki page per release in the SPB Gitlab instance with a mapping between
  105 +strategical, tactical, and operational views. In a practical point of view, one
  106 +milestone per user history (feature), and one or more issues for addressing
  107 +each feature. With this approach we achieved two important goals: keeping all
  108 +the management as close possible to to the source code, and tracking every
  109 +feature developed during the project. Our decision to
  110 +use Wiki initially was empirical, but later reinforced by a research conducted
  111 +by Joseph Chao showing the advantage of using Wikis \cite{chao2007student,
  112 +opensourcestyle}.
  113 +
  114 +\subsection{High-level project management and reporting}
  115 +
  116 +The Brazilian government used to work with software development in a
  117 +very traditional way. They would frequently focus on documents and not
  118 +on what was, in our opinion, what really matters: working software. This
  119 +dissonance caused us a communication noise with MP, because they would
  120 +often question our work style. It was especially hard to convince them
  121 +to accept the idea of open scope and agile development, but after months
  122 +of labor and showing results they stopped resisting.
  123 +
  124 +We defined some level of meeting granularity to avoid generating too
  125 +much overhead to the developers. We had a strategical and validating
  126 +meeting with MP (the former once in a month and the latter each 15th
  127 +day), release plaining with the entire team (one per month), and finally
  128 +a sprint planning (one each 15th day). Figure \ref{fig:meeting} is a
  129 +diagram that represents our meeting organization.
  130 +
  131 +\begin{figure}[hbt]
  132 + \centering
  133 + \includegraphics[width=\linewidth]{figures/meeting_flows.png}
  134 + \caption{Meetings cycles.}
  135 + \label{fig:meeting}
  136 +\end{figure}
  137 +
  138 +In the strategical meeting we usually defined the priorities and new
  139 +features with the Brazilian government. Normally the professors, the
  140 +coach of each team, the meta-coach, and some employees of the MP would
  141 +participate in this meeting. We usually discussed what the team already
  142 +produced since our last meeting, and established the new features for
  143 +the next release. Notice that just part of the team would join this
  144 +meeting to avoid generating unnecessary overhead to the developers, but
  145 +all the students interested to participate were allowed to join (many
  146 +students wanted this experience during the project).
  147 +
  148 +After the strategical meeting with Brazilian government agents, we had a
  149 +planning turn with all teams together. In this part, each team worked together
  150 +to convert the MP wishes into smaller parts which were represented by the epics of
  151 +the release. Each coach was responsible for conducting the planning, and after
  152 +that record it on the project wiki (the wiki provided by Gitlab). With this
  153 +epic, each 14th day the team have generated their sprint scheduler (with small
  154 +achievements mapped to issues).
  155 +
  156 +To keep the Brazilian government always updated, we invited them to work
  157 +with us to validate the new features in progress. Normally we had a
  158 +meeting each 15th day. Basically, this was our work flow, we always kept
  159 +everything extremely open to the MP (our way of working, and the one
  160 +often used by open source projects) and to the team.
  161 +
  162 +To keep the track of all of those things we used the SPB itself,
  163 +especially Gitlab. Basically, we had:
  164 +
  165 +\begin{enumerate}
  166 + \item Project repository: We have one organization with many repositories;
  167 + \item Milestones: Each milestone was used to register a release;
  168 + \item Wiki: Each release has one page on wiki with the compilation of
  169 + strategical meeting;
  170 + \item Issues: Each sprint planning generated issues, which we associated to
  171 + the specific milestone and updated the wiki with the issue number related
  172 + with them. Finally each developer assigned the issue to itself.
  173 +\end{enumerate}
  174 +
  175 +Notice that this workflow gave us and the Brazilian government agents full
  176 +traceability from a high level view of each feature to the lowest level (code).
  177 +It is important to highlight that we converge to this workflow after many
  178 +experiments. For example, we used a tool named Redmine for organizing our tasks
  179 +during the sprints, however, this tool revealed inefficient for our case since
  180 +the government agents lost part of the project traceability. We realized that
  181 +centralize all the work in the SPB portal is the best option for our case.
... ...
opensym2017/content/08-ux.tex
View file @ca3c632
... ... @@ -1,34 +0,0 @@
1   -\section{User eXperience evolution}
2   -\label{sec:ux}
3   -
4   -The integration of collaborative environments goes beyond functional aspects.
5   -Offering the population an unified experience across these environments has
6   -been the key to encourage the use of the platform as it reduces the perception
7   -of complexity. Thus, the SPB Portal information architecture was redesigned
8   -to provide a transparent navigation and to reach users with different profiles.
9   -A process of harmonization has been employed on the interaction models of each
10   -tool to reduce the learning curve. At the same time, a new visual style was
11   -created to unify the navigation experience and to comply with the guidelines of
12   -the digital communication identity standard established by the Federal
13   -Government.
14   -
15   -With the increase in system features and the addition of new tools, the
16   -visual style has steadily evolved to keep the navigation unified. Moreover,
17   -tools from different backgrounds, which in many cases provide similar
18   -functionality, prompted the development of an unified interface. Some
19   -features, such as search and user profile editing were eliminated from
20   -the individual applications, and implemented centrally to ensure a
21   -consistent look and feel.
22   -
23   -Another challenge was responsive web design. The integrated applications
24   -had varying degrees of support for responsiveness, and the common
25   -interface had to adapt for each individual scenario. In particular
26   -Noosfero did not yet have a responsive design; we engaged in its
27   -development and contributed towards that goal.
28   -
29   -After the initial release of the new SPB Portal in 2014, several
30   -validations activities were implemented in 2015 and 2016. The aim was to
31   -provide the most wanted features by casual users (such as public
32   -servants interested in downloads and documentation) immediately, while
33   -allowing more experienced users (such as developers) to easily drill down
34   -to the details.
opensym2017/content/09-conclusion.tex 0 → 100644
  Diff comments   View file @5588ee4
... ... @@ -0,0 +1,220 @@
  1 +\section{Conclusion}
  2 +\label{sec:conclusion}
  3 +
  4 +In this work, we presented and discussed issues experienced during a government-funded
  5 +project, in partnership with the University of Brasilia and the University of
  6 +São Paulo, to evolve the Brazilian Public Software Portal.
  7 +Its contributions are twofold. First, we present the strategy used to develop
  8 +and to deliver an unprecedented platform to Brazilian government. Second,
  9 +based on the results of the SPB Portal project, we point out that it is
  10 +possible to mitigate conflicts experienced in the development environment
  11 +and to conciliate governmental and academy cultures. To summarize our main contributions, we answered in this section the three open questions those guided this paper.
  12 +
  13 +
  14 +\textbf{Q1 -- Which strategy could be used to integrate several existing
  15 +FLOSS tools to promote the collaborative software development?}
  16 +%
  17 +The SPB portal integrates more than 10 FLOSS tools and provides several features,
  18 +such as social network, mailing list, version control, content management and
  19 +source code quality monitoring. Concerned with the platform sustainability and
  20 +maintainability, the aforementioned 10 FLOSS tools were integrated with minimum
  21 +differences from their official versions and the new developed features were
  22 +sent upstream to ensure an alignment between the portal systems and their
  23 +respective official versions. In the integration process, the main softwares
  24 +were identified, specific teams were formed to work with each one of them
  25 +and each team was composed of students with different levels of skills and at
  26 +least one senior professional.
  27 +
  28 +\textbf{Q2 -- How to involve students in real-world projects interacting with
  29 +real customers?}
  30 +%
  31 +In terms of mitigating conflicts, we tried to show that, as long as the
  32 +university can provide a healthy and challenging environment to its students,
  33 +one may conciliate studies and professional training in universities.
  34 +%
  35 +The students interacted with professionals of diverse fields of expertise, and they were
  36 +able to participate in all levels of the software development process.
  37 +This contributed to a great learning opportunity.
  38 +%
  39 +In our work process, based on open and collaborative software development
  40 +practices, students could negotiate their work schedule as well as count on IT
  41 +professionals to solve development issues.
  42 +%
  43 +Among the students, we have defined coaches for each team and a meta-coach
  44 +(coach of whole project). All coaches, together with professors, have
  45 +intermediated the communication between client (Ministry of Planning of Brazil)
  46 +and the rest of the group.
  47 +After the end of the project, some students successfully
  48 +embraced opportunities in public and private sectors, within national borders
  49 +and abroad. Some other students went further and started their own companies.
  50 +
  51 +\textbf{Q3 -- How to introduce the FLOSS collaborative and agile
  52 +practices to governmental development process?}
  53 +With some adaptations, what we called the ``translation processes'', it is feasible
  54 +to conciliate agile methodologies and FLOSS practices to develop software to
  55 +governmental organizations with functional hierarchical structures that use
  56 +traditional development paradigm.
  57 +%
  58 +Aiming at reducing client questions about workconclusion, a DevOps front was
  59 +created to automate all deploy process and also to work in continuous
  60 +delivery. The government was brought to our work environment and interacted
  61 +with our management and communication tools. For the project success, we
  62 +focused on providing a friendly working environment as well as on showing to
  63 +governmental agents another way to interact with the FLOSS community and the
  64 +university.
  65 +
  66 +\subsection{Lessons Learned}
  67 +\label{sec:lessons}
  68 +
  69 +From the answers of our initial open questions, we can also highlighted six lessons learned to make easier to share our experience during the development of the new SPB Portal.
  70 +
  71 +\textbf{L1 -- The participation of experienced professionals is crucial to
  72 +the success of the project.}
  73 +One important factor for the students was the composition of the teams
  74 +with the participation of experienced professionals.
  75 +%
  76 +On the technical side, the senior developers and designers would handle
  77 +the more difficult technical decisions, creating a work environment
  78 +where the students could develop their skills in a didactic way without
  79 +pressure.
  80 +%
  81 +On the management side, the active participation of professors -- who
  82 +are, in the end, the ones responsible for the project -- is crucial to
  83 +make sure students participation is conducted in a healthy way, and it is an
  84 +instance of leading by example.
  85 +
  86 +\textbf{L2 -- A balanced relationship between academia and industry.}
  87 +The experience of the SPB project led UnB to develop a work style which
  88 +proved to be appropriate for an educational environment that brings
  89 +academia and industry together.
  90 +%
  91 +The highest priority from the university's point of view is the
  92 +students. Considering this, the activities of the project were never
  93 +prioritized to the detriment of classes and other pedagogical
  94 +activities. In summary, we had students working at different times, part
  95 +time, remotely or locally, always respecting their individual
  96 +conditions, but doing the work in a collective, collaborative and open
  97 +way.
  98 +%
  99 +And even under a potentially adverse environment, the project delivered
  100 +the desired solution with success.
  101 +%
  102 +At the end of the project, we noted that the skills developed by the
  103 +students were at the software engineering state of the art.
  104 +After the project ended, we had team members successfully embracing
  105 +opportunities in public, private, national, and international
  106 +organizations, in addition to those students that
  107 +opened their own companies.
  108 +
  109 +\textbf{L3 -- Managing different organizational cultures.}
  110 +In the beginning of the project, the Brazilian Government stakeholders
  111 +had certain expectations about the project development that, let's
  112 +say, didn't exactly match our work style based on agile and FLOSS practices.
  113 +%
  114 +We had to develop strategies that would support these different
  115 +organizational cultures. The
  116 +MP is organized in a functional hierarchical organizational structure,
  117 +typically adopting a traditional development paradigm. Therefore, we had to
  118 +create a translation process between our team and the MP managers who
  119 +managed the project on their side assuming a traditional waterfall
  120 +process.
  121 +
  122 +\textbf{L4 -- Managing higher-level and lower-level goals separately.}
  123 +During the initial phase of the project, the MP team has brought
  124 +strategic discussions to technical/operational meetings that
  125 +were supposed to be about practical technical decisions.
  126 +%
  127 +This produced a highly complex communication and management environment,
  128 +overloading the professors because they were supposed
  129 +for maintaining the MP strategy synchronized with the
  130 +implementation plans of the development team. This was hard, especially because the
  131 +aforementioned cultural mismatch. Mixing both concerns in the same
  132 +discussions caused confusion on both sides.
  133 +%
  134 +From the middle of the project we were able to keep those
  135 +concerns separated, what eased the work of everyone involved.
  136 +
  137 +\textbf{L5 -- Living with ill-advised political decisions.}
  138 +At the initial phases of the project, by political and personal
  139 +motivation, the main stakeholders from the Brazilian government imposed
  140 +the use of Colab to guide the development of the new SPB platform. Our
  141 +team was totally against the idea because we already knew that Colab was
  142 +a very experimental project and its adoption could dramatically increase
  143 +the project complexity. Even though we provided technical reasons to
  144 +not utilize Colab, the MP was adamant and we had to manage this problem. We did massive changes to
  145 +Colab, and at the end of the project we have completely rewritten it to make
  146 +it stable. It is important to notice that the MP compelled us to accept
  147 +a technical decision based only on political interests, without considering
  148 +all the resources that would be spent due to that decision. At the end
  149 +of the project, we verified that Colab indeed consumed a vast amount of
  150 +the budget and increased the project complexity. At the end of the
  151 +project and after our analysis on the decision made by the MP, we
  152 +understand that MP managers are capable of ignoring technical reasons
  153 +in favor of political decisions.
  154 +
  155 +\textbf{L6 -- Consider sustainability from the beginning.}
  156 +In the process of deploying the SPB platform in the MP infrastructure we had
  157 +to interact with the MP technicians. We did several workshops, training
  158 +and a meticulous documentation describing all the required procedures to
  159 +update the platform, however, we realized that the MP technicians
  160 +constantly avoid the responsibility. After noticing the aforementioned
  161 +situation, we organized a DevOps team that completely automated all
  162 +the deployment procedure. We simplified all the platform deployment to a
  163 +few steps: (1) initial configurations (just ssh configuration) and (2)
  164 +the execution of simple commands to completely update the platform. By
  165 +the end of the project, we observed that the MP technicians invariably
  166 +still depended on our support to update the platform even with all the
  167 +automation provided by us. We were sadly left with a feeling of
  168 +uncertainty about the future of the platform after the project ended. In
  169 +hindsight, we realize that the MP dedicated system analysts and
  170 +managers to the project, but not operations technicians. The later
  171 +should have been more involved with the process so they could at least be
  172 +comfortable in managing the platform infrastructure.
  173 +
  174 +
  175 +\textbf{Final remarks and future work}
  176 +
  177 +The portal is available at \url{softwarepublico.gov.br}. All
  178 +documentation, including detailed architecture and operation manuals are
  179 +also available\footnote{\url{https://softwarepublico.gov.br/doc/}
  180 +(in Portuguese only at the moment)}.
  181 +%
  182 +All the integrated tools are FLOSS and our contributions were published
  183 +in open repositories, available on the SPB Portal itself. We also
  184 +contributed these features back to the respective communities, which
  185 +benefits both those communities and us, since we can share future
  186 +development and maintenance effort with other organizations that
  187 +participate in these projects.
  188 +
  189 +Future work should use data produced by the project to validate and evaluate
  190 +how the used FLOSS and Agile practices have impacted the students and the
  191 +governmental development process. For this, we would conduce a \textit{postmortem}
  192 +analysis using the project open data and a survey targeting the involved actors.
  193 +
  194 +%===========
  195 +% Conclusion
  196 +%===========
  197 +
  198 +% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
  199 +%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
  200 +%% afirmação, embora eu e Paulo consigamos perceber isso.
  201 +
  202 +
  203 +%* utilização do projeto para formação de recursos humanos (alunos)
  204 +
  205 +%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
  206 +
  207 +%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
  208 +
  209 +%* 69 projetos marcados como SPB, de 81 no total na plataforma.
  210 +
  211 +%* 47\% é desenvolvido em PHP.
  212 +
  213 +% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
  214 +
  215 +% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
  216 +
  217 +% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
  218 +
  219 +%* Debater economia de recursos em orgão públicos
  220 +
... ...
opensym2017/content/09-process.tex
View file @ca3c632
... ... @@ -1,181 +0,0 @@
1   -\section{Development Organization and Process}
2   -\label{sec:process}
3   -
4   -The SPB team was composed of a variety of professionals with different levels
5   -and skills, where most of them were undergraduate students with major in
6   -software engineering (from 4th semester or upper). Since the students could
7   -not dedicate many hours per week to the project, they always had the
8   -flexibility to negotiate their work schedule during the semester in
9   -order to not harm their classes and coursework. Their daily work routine
10   -in the project included programming and DevOps tasks.
11   -
12   -The development of SPB project required a vast experience and background that
13   -usually undergraduate students do not have yet. For this reason, a few senior
14   -developers have joined to the project to help with the more difficult issues
15   -and to transfer knowledge to the students. Their main task was to provide
16   -solutions for complex problems, in other words, they worked as developers. As
17   -these professionals are very skillful and the project could not fund a full
18   -time work for them, some of them worked partially on the project. In addition,
19   -they lived in a different states spread around Brazil which led much of the
20   -communication to be online.
21   -
22   -In short, our work process was based on open and collaborative software
23   -development practices. The development process was defined based on the
24   -adaptation of different agile and FLOSS communities practices, highlighting the
25   -high degree of automation resulting from DevOps practices. Thus, the work
26   -process was executed in a cadenced and continuous way.
27   -
28   -Finally, the last group of actors of this project was composed of employees
29   -formally working for the Brazilian government, in the Ministry of Planning,
30   -Development, and Management (MP is the Brazilian acronym). All the project
31   -decisions, validations, and scope definitions were made by them. In this way we
32   -developed software product incrementally, with releases aligned to business
33   -strategic objectives. As one can see, the project had many different
34   -sort of stakeholders that had to be organized and synchronized.
35   -
36   -\subsection{Team organization}
37   -
38   -Approximately 70\% of the development teams were composed of software
39   -engineering undergraduate students from UnB and they worked physically
40   -in the same laboratory. Each student had their own schedule based on
41   -their classes, what complicates the implementation of pair programming.
42   -The senior developers tried to synchronize their schedule with the
43   -students on their sub-team. To cope with this environment, we had a few
44   -basic rules which guided the project organization:
45   -
46   -\begin{enumerate}
47   - \item Classes have high priority for undergraduate students;
48   - \item Work in pairs whenever possible (locally or remotely);
49   - \item There must be one morning or afternoon per week when
50   - \emph{everyone} should be together physically in the laboratory
51   - (except of course the remote team members);
52   - \item Every 3 to 4 months, the senior developers would fly in and work
53   - alongside the students for a few days.
54   -\end{enumerate}
55   -
56   -With the aforementioned rules we divided all the project into four
57   -different teams: Colab, Noosfero, Design, and DevOps. Each team had one
58   -coach responsible for reducing the communication problem with the other
59   -teams and help the members to organize themselves in the best way for
60   -everyone (always respecting their work time). The coach was one of the
61   -students working in some of the teams with the extra duty of registering
62   -the current tasks developed in the sprint and with the responsibility to
63   -talk with other teams. One important thing to notice is the mutability
64   -of the team and the coach, during the project many students changed
65   -between the teams to try different areas.
66   -
67   -One characteristic of the teams was the presence of (at least) one
68   -senior per team. This was essential, because hard decisions and complex
69   -problems were usually referred to them. This kept having to take
70   -complicated technical decisions from the coach tole, what encouraged
71   -students to be coaches. Lastly, the senior developers worked directly
72   -with the students, and this was important to give the undergraduate the
73   -opportunity to interact with a savvy professional in his area and to
74   -keep the knowledge flowing in the project.
75   -
76   -Finally, we had to add two last elements of the team organization, that
77   -was essential for the project harmony: the meta-coach and professors.
78   -The former was a software engineer recently graduated and which wanted
79   -to keep working on the project, the latter were professors that
80   -orchestrated all the interactions between all members of the project.
81   -The meta-coach usually worked in one specific team and had the extra
82   -task of knowing the current status of all teams. Professors and
83   -meta-coaches worked together to reduce the communication problem between
84   -all the teams. Lastly, all the paperwork tasks, such as reporting on the
85   -project progress to the Ministry, was handled by the professors.
86   -
87   -\subsection{Communication and management}
88   -
89   -Our team had many people working together, and most of the seniors worked in a
90   -different city remotely. Also, we tried to keep our work completely clear to
91   -the Brazilian government and citizens interested in following the project. To
92   -handle those cases, we used a set of tools to communication and other to manage
93   -the project.
94   -
95   -For communication between members in different places, we used: video
96   -conferencing with shared terminal tools, IRC, and mailing lists. For example,
97   -when one student had to work in pair with a senior, normally, they used video
98   -conferencing tool for talking and shared a terminal session (both typing and
99   -seeing the screen in real time). For questions and fast discussion, we used
100   -IRC. For general notification, we used the mailing lists.
101   -
102   -For managing the project we used the SPB Portal itself; first to validate it by
103   -ourselves, and also because it had all the required tools. We basically created
104   -one wiki page per release in the SPB Gitlab instance with a mapping between
105   -strategical, tactical, and operational views. In a practical point of view, one
106   -milestone per user history (feature), and one or more issues for addressing
107   -each feature. With this approach we achieved two important goals: keeping all
108   -the management as close possible to to the source code, and tracking every
109   -feature developed during the project. Our decision to
110   -use Wiki initially was empirical, but later reinforced by a research conducted
111   -by Joseph Chao showing the advantage of using Wikis \cite{chao2007student,
112   -opensourcestyle}.
113   -
114   -\subsection{High-level project management and reporting}
115   -
116   -The Brazilian government used to work with software development in a
117   -very traditional way. They would frequently focus on documents and not
118   -on what was, in our opinion, what really matters: working software. This
119   -dissonance caused us a communication noise with MP, because they would
120   -often question our work style. It was especially hard to convince them
121   -to accept the idea of open scope and agile development, but after months
122   -of labor and showing results they stopped resisting.
123   -
124   -We defined some level of meeting granularity to avoid generating too
125   -much overhead to the developers. We had a strategical and validating
126   -meeting with MP (the former once in a month and the latter each 15th
127   -day), release plaining with the entire team (one per month), and finally
128   -a sprint planning (one each 15th day). Figure \ref{fig:meeting} is a
129   -diagram that represents our meeting organization.
130   -
131   -\begin{figure}[hbt]
132   - \centering
133   - \includegraphics[width=\linewidth]{figures/meeting_flows.png}
134   - \caption{Meetings cycles.}
135   - \label{fig:meeting}
136   -\end{figure}
137   -
138   -In the strategical meeting we usually defined the priorities and new
139   -features with the Brazilian government. Normally the professors, the
140   -coach of each team, the meta-coach, and some employees of the MP would
141   -participate in this meeting. We usually discussed what the team already
142   -produced since our last meeting, and established the new features for
143   -the next release. Notice that just part of the team would join this
144   -meeting to avoid generating unnecessary overhead to the developers, but
145   -all the students interested to participate were allowed to join (many
146   -students wanted this experience during the project).
147   -
148   -After the strategical meeting with Brazilian government agents, we had a
149   -planning turn with all teams together. In this part, each team worked together
150   -to convert the MP wishes into smaller parts which were represented by the epics of
151   -the release. Each coach was responsible for conducting the planning, and after
152   -that record it on the project wiki (the wiki provided by Gitlab). With this
153   -epic, each 14th day the team have generated their sprint scheduler (with small
154   -achievements mapped to issues).
155   -
156   -To keep the Brazilian government always updated, we invited them to work
157   -with us to validate the new features in progress. Normally we had a
158   -meeting each 15th day. Basically, this was our work flow, we always kept
159   -everything extremely open to the MP (our way of working, and the one
160   -often used by open source projects) and to the team.
161   -
162   -To keep the track of all of those things we used the SPB itself,
163   -especially Gitlab. Basically, we had:
164   -
165   -\begin{enumerate}
166   - \item Project repository: We have one organization with many repositories;
167   - \item Milestones: Each milestone was used to register a release;
168   - \item Wiki: Each release has one page on wiki with the compilation of
169   - strategical meeting;
170   - \item Issues: Each sprint planning generated issues, which we associated to
171   - the specific milestone and updated the wiki with the issue number related
172   - with them. Finally each developer assigned the issue to itself.
173   -\end{enumerate}
174   -
175   -Notice that this workflow gave us and the Brazilian government agents full
176   -traceability from a high level view of each feature to the lowest level (code).
177   -It is important to highlight that we converge to this workflow after many
178   -experiments. For example, we used a tool named Redmine for organizing our tasks
179   -during the sprints, however, this tool revealed inefficient for our case since
180   -the government agents lost part of the project traceability. We realized that
181   -centralize all the work in the SPB portal is the best option for our case.
opensym2017/content/10-contributions.tex
View file @ca3c632
... ... @@ -1,58 +0,0 @@
1   -\section{Contributions to the upstream communities}
2   -\label{sec:contributions}
3   -
4   -%- projeto feito do jeito certo com relação ao software livre (contribuições upstream etc)
5   -%* Colab -> RevProxy
6   -%* Colab, atualização do python/django
7   -%* Contribuições para o GitLab (autenticação)
8   -%* Noosfero, atualização do Rails, preparação para federação, nova interface ...
9   -%* Coper, empacotamentos (obs), omniauth
10   -
11   -
12   -During the execution of this project we made several contributions to
13   -the upstream communities we interacted with. This occurred due to our
14   -development process aligned with those of the respective communities.
15   -During development, we would explicitly discuss the features and bug
16   -fixes that we were working on with the applicable upstream communities.
17   -This contributed to improve the developers technical solutions with
18   -expertise outside of our team, and make it easier for those changes to
19   -be accepted in the original projects. Having changes accepted upstream
20   -in turn makes our life easier as it minimizes the delta between our
21   -codebase and upstream's allowing us to upgrade and benefit from
22   -development work from others.
23   -
24   -In Colab, we helped upstream redesign the entirely architecture,
25   -enabling the development of plugins to integrate new tools. We also
26   -added a feature that allowed Colab to run asynchronous tasks, which was
27   -a major improvement for us since we were developing a complex system. A
28   -migration to the latest Django version was made (web framework used by
29   -Colab). Moreover, we worked on RevProxy (the more important dependency
30   -of Colab) to put it in a good shape, fixing many bugs.
31   -
32   -Gitlab was the tool that we made the least number of modifications. We
33   -contributed with some improvements related with configuration files and
34   -we developed a new plugin that enables user authentication in Gitlab
35   -through the REMOTE\_USER HTTP header. This plugin was needed because
36   -Colab uses this mechanism to manage the authentication.
37   -
38   -Noosfero was the tool that contemplated several functional requirements,
39   -therefore we made a large number of contributions with upstream. We helped to
40   -migrate to the latest Rails version (web framework used by Noosfero), enable
41   -the federation implementation (federation with other social networks), and
42   -decouple the interface and the back-end.
43   -
44   -Mezuro was completely rewritten and its architecture evolved adopting the
45   -microservice architecture\cite{namiot2014micro}. This way, we minimize the
46   -amount of code to maintain it, helping to test it and grant its code quality.
47   -In practice, we modularize the Mezuro project in several independent services.
48   -Currently, its computation and visualization modules use Kalibro and Prezento,
49   -respectively. They were developed into the Mezuro project and evolved during
50   -its integration to the new SPB Portal.
51   -
52   -
53   -We also made some contributions on the DevOps front. Some members of
54   -them team became maintainers of some Python libraries that were used by
55   -our scripts to upload packages to OBS (Open Build Service). We developed
56   -a tool called copr-status to keep track of the different stages of the
57   -lifecycle of each of the individual packages we were working on.
58   -
opensym2017/content/11-conclusion.tex
View file @ca3c632
... ... @@ -1,220 +0,0 @@
1   -\section{Conclusion}
2   -\label{sec:conclusion}
3   -
4   -In this work, we presented and discussed issues experienced during a government-funded
5   -project, in partnership with the University of Brasilia and the University of
6   -São Paulo, to evolve the Brazilian Public Software Portal.
7   -Its contributions are twofold. First, we present the strategy used to develop
8   -and to deliver an unprecedented platform to Brazilian government. Second,
9   -based on the results of the SPB Portal project, we point out that it is
10   -possible to mitigate conflicts experienced in the development environment
11   -and to conciliate governmental and academy cultures. To summarize our main contributions, we answered in this section the three open questions those guided this paper.
12   -
13   -
14   -\textbf{Q1 -- Which strategy could be used to integrate several existing
15   -FLOSS tools to promote the collaborative software development?}
16   -%
17   -The SPB portal integrates more than 10 FLOSS tools and provides several features,
18   -such as social network, mailing list, version control, content management and
19   -source code quality monitoring. Concerned with the platform sustainability and
20   -maintainability, the aforementioned 10 FLOSS tools were integrated with minimum
21   -differences from their official versions and the new developed features were
22   -sent upstream to ensure an alignment between the portal systems and their
23   -respective official versions. In the integration process, the main softwares
24   -were identified, specific teams were formed to work with each one of them
25   -and each team was composed of students with different levels of skills and at
26   -least one senior professional.
27   -
28   -\textbf{Q2 -- How to involve students in real-world projects interacting with
29   -real customers?}
30   -%
31   -In terms of mitigating conflicts, we tried to show that, as long as the
32   -university can provide a healthy and challenging environment to its students,
33   -one may conciliate studies and professional training in universities.
34   -%
35   -The students interacted with professionals of diverse fields of expertise, and they were
36   -able to participate in all levels of the software development process.
37   -This contributed to a great learning opportunity.
38   -%
39   -In our work process, based on open and collaborative software development
40   -practices, students could negotiate their work schedule as well as count on IT
41   -professionals to solve development issues.
42   -%
43   -Among the students, we have defined coaches for each team and a meta-coach
44   -(coach of whole project). All coaches, together with professors, have
45   -intermediated the communication between client (Ministry of Planning of Brazil)
46   -and the rest of the group.
47   -After the end of the project, some students successfully
48   -embraced opportunities in public and private sectors, within national borders
49   -and abroad. Some other students went further and started their own companies.
50   -
51   -\textbf{Q3 -- How to introduce the FLOSS collaborative and agile
52   -practices to governmental development process?}
53   -With some adaptations, what we called the ``translation processes'', it is feasible
54   -to conciliate agile methodologies and FLOSS practices to develop software to
55   -governmental organizations with functional hierarchical structures that use
56   -traditional development paradigm.
57   -%
58   -Aiming at reducing client questions about workconclusion, a DevOps front was
59   -created to automate all deploy process and also to work in continuous
60   -delivery. The government was brought to our work environment and interacted
61   -with our management and communication tools. For the project success, we
62   -focused on providing a friendly working environment as well as on showing to
63   -governmental agents another way to interact with the FLOSS community and the
64   -university.
65   -
66   -\subsection{Lessons Learned}
67   -\label{sec:lessons}
68   -
69   -From the answers of our initial open questions, we can also highlighted six lessons learned to make easier to share our experience during the development of the new SPB Portal.
70   -
71   -\textbf{L1 -- The participation of experienced professionals is crucial to
72   -the success of the project.}
73   -One important factor for the students was the composition of the teams
74   -with the participation of experienced professionals.
75   -%
76   -On the technical side, the senior developers and designers would handle
77   -the more difficult technical decisions, creating a work environment
78   -where the students could develop their skills in a didactic way without
79   -pressure.
80   -%
81   -On the management side, the active participation of professors -- who
82   -are, in the end, the ones responsible for the project -- is crucial to
83   -make sure students participation is conducted in a healthy way, and it is an
84   -instance of leading by example.
85   -
86   -\textbf{L2 -- A balanced relationship between academia and industry.}
87   -The experience of the SPB project led UnB to develop a work style which
88   -proved to be appropriate for an educational environment that brings
89   -academia and industry together.
90   -%
91   -The highest priority from the university's point of view is the
92   -students. Considering this, the activities of the project were never
93   -prioritized to the detriment of classes and other pedagogical
94   -activities. In summary, we had students working at different times, part
95   -time, remotely or locally, always respecting their individual
96   -conditions, but doing the work in a collective, collaborative and open
97   -way.
98   -%
99   -And even under a potentially adverse environment, the project delivered
100   -the desired solution with success.
101   -%
102   -At the end of the project, we noted that the skills developed by the
103   -students were at the software engineering state of the art.
104   -After the project ended, we had team members successfully embracing
105   -opportunities in public, private, national, and international
106   -organizations, in addition to those students that
107   -opened their own companies.
108   -
109   -\textbf{L3 -- Managing different organizational cultures.}
110   -In the beginning of the project, the Brazilian Government stakeholders
111   -had certain expectations about the project development that, let's
112   -say, didn't exactly match our work style based on agile and FLOSS practices.
113   -%
114   -We had to develop strategies that would support these different
115   -organizational cultures. The
116   -MP is organized in a functional hierarchical organizational structure,
117   -typically adopting a traditional development paradigm. Therefore, we had to
118   -create a translation process between our team and the MP managers who
119   -managed the project on their side assuming a traditional waterfall
120   -process.
121   -
122   -\textbf{L4 -- Managing higher-level and lower-level goals separately.}
123   -During the initial phase of the project, the MP team has brought
124   -strategic discussions to technical/operational meetings that
125   -were supposed to be about practical technical decisions.
126   -%
127   -This produced a highly complex communication and management environment,
128   -overloading the professors because they were supposed
129   -for maintaining the MP strategy synchronized with the
130   -implementation plans of the development team. This was hard, especially because the
131   -aforementioned cultural mismatch. Mixing both concerns in the same
132   -discussions caused confusion on both sides.
133   -%
134   -From the middle of the project we were able to keep those
135   -concerns separated, what eased the work of everyone involved.
136   -
137   -\textbf{L5 -- Living with ill-advised political decisions.}
138   -At the initial phases of the project, by political and personal
139   -motivation, the main stakeholders from the Brazilian government imposed
140   -the use of Colab to guide the development of the new SPB platform. Our
141   -team was totally against the idea because we already knew that Colab was
142   -a very experimental project and its adoption could dramatically increase
143   -the project complexity. Even though we provided technical reasons to
144   -not utilize Colab, the MP was adamant and we had to manage this problem. We did massive changes to
145   -Colab, and at the end of the project we have completely rewritten it to make
146   -it stable. It is important to notice that the MP compelled us to accept
147   -a technical decision based only on political interests, without considering
148   -all the resources that would be spent due to that decision. At the end
149   -of the project, we verified that Colab indeed consumed a vast amount of
150   -the budget and increased the project complexity. At the end of the
151   -project and after our analysis on the decision made by the MP, we
152   -understand that MP managers are capable of ignoring technical reasons
153   -in favor of political decisions.
154   -
155   -\textbf{L6 -- Consider sustainability from the beginning.}
156   -In the process of deploying the SPB platform in the MP infrastructure we had
157   -to interact with the MP technicians. We did several workshops, training
158   -and a meticulous documentation describing all the required procedures to
159   -update the platform, however, we realized that the MP technicians
160   -constantly avoid the responsibility. After noticing the aforementioned
161   -situation, we organized a DevOps team that completely automated all
162   -the deployment procedure. We simplified all the platform deployment to a
163   -few steps: (1) initial configurations (just ssh configuration) and (2)
164   -the execution of simple commands to completely update the platform. By
165   -the end of the project, we observed that the MP technicians invariably
166   -still depended on our support to update the platform even with all the
167   -automation provided by us. We were sadly left with a feeling of
168   -uncertainty about the future of the platform after the project ended. In
169   -hindsight, we realize that the MP dedicated system analysts and
170   -managers to the project, but not operations technicians. The later
171   -should have been more involved with the process so they could at least be
172   -comfortable in managing the platform infrastructure.
173   -
174   -
175   -\textbf{Final remarks and future work}
176   -
177   -The portal is available at \url{softwarepublico.gov.br}. All
178   -documentation, including detailed architecture and operation manuals are
179   -also available\footnote{\url{https://softwarepublico.gov.br/doc/}
180   -(in Portuguese only at the moment)}.
181   -%
182   -All the integrated tools are FLOSS and our contributions were published
183   -in open repositories, available on the SPB Portal itself. We also
184   -contributed these features back to the respective communities, which
185   -benefits both those communities and us, since we can share future
186   -development and maintenance effort with other organizations that
187   -participate in these projects.
188   -
189   -Future work should use data produced by the project to validate and evaluate
190   -how the used FLOSS and Agile practices have impacted the students and the
191   -governmental development process. For this, we would conduce a \textit{postmortem}
192   -analysis using the project open data and a survey targeting the involved actors.
193   -
194   -%===========
195   -% Conclusion
196   -%===========
197   -
198   -% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
199   -%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
200   -%% afirmação, embora eu e Paulo consigamos perceber isso.
201   -
202   -
203   -%* utilização do projeto para formação de recursos humanos (alunos)
204   -
205   -%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
206   -
207   -%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
208   -
209   -%* 69 projetos marcados como SPB, de 81 no total na plataforma.
210   -
211   -%* 47\% é desenvolvido em PHP.
212   -
213   -% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
214   -
215   -% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
216   -
217   -% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
218   -
219   -%* Debater economia de recursos em orgão públicos
220   -
opensym2017/content/out-contributions.tex 0 → 100644
  Diff comments   View file @5588ee4
... ... @@ -0,0 +1,58 @@
  1 +\section{Contributions to the upstream communities}
  2 +\label{sec:contributions}
  3 +
  4 +%- projeto feito do jeito certo com relação ao software livre (contribuições upstream etc)
  5 +%* Colab -> RevProxy
  6 +%* Colab, atualização do python/django
  7 +%* Contribuições para o GitLab (autenticação)
  8 +%* Noosfero, atualização do Rails, preparação para federação, nova interface ...
  9 +%* Coper, empacotamentos (obs), omniauth
  10 +
  11 +
  12 +During the execution of this project we made several contributions to
  13 +the upstream communities we interacted with. This occurred due to our
  14 +development process aligned with those of the respective communities.
  15 +During development, we would explicitly discuss the features and bug
  16 +fixes that we were working on with the applicable upstream communities.
  17 +This contributed to improve the developers technical solutions with
  18 +expertise outside of our team, and make it easier for those changes to
  19 +be accepted in the original projects. Having changes accepted upstream
  20 +in turn makes our life easier as it minimizes the delta between our
  21 +codebase and upstream's allowing us to upgrade and benefit from
  22 +development work from others.
  23 +
  24 +In Colab, we helped upstream redesign the entirely architecture,
  25 +enabling the development of plugins to integrate new tools. We also
  26 +added a feature that allowed Colab to run asynchronous tasks, which was
  27 +a major improvement for us since we were developing a complex system. A
  28 +migration to the latest Django version was made (web framework used by
  29 +Colab). Moreover, we worked on RevProxy (the more important dependency
  30 +of Colab) to put it in a good shape, fixing many bugs.
  31 +
  32 +Gitlab was the tool that we made the least number of modifications. We
  33 +contributed with some improvements related with configuration files and
  34 +we developed a new plugin that enables user authentication in Gitlab
  35 +through the REMOTE\_USER HTTP header. This plugin was needed because
  36 +Colab uses this mechanism to manage the authentication.
  37 +
  38 +Noosfero was the tool that contemplated several functional requirements,
  39 +therefore we made a large number of contributions with upstream. We helped to
  40 +migrate to the latest Rails version (web framework used by Noosfero), enable
  41 +the federation implementation (federation with other social networks), and
  42 +decouple the interface and the back-end.
  43 +
  44 +Mezuro was completely rewritten and its architecture evolved adopting the
  45 +microservice architecture\cite{namiot2014micro}. This way, we minimize the
  46 +amount of code to maintain it, helping to test it and grant its code quality.
  47 +In practice, we modularize the Mezuro project in several independent services.
  48 +Currently, its computation and visualization modules use Kalibro and Prezento,
  49 +respectively. They were developed into the Mezuro project and evolved during
  50 +its integration to the new SPB Portal.
  51 +
  52 +
  53 +We also made some contributions on the DevOps front. Some members of
  54 +them team became maintainers of some Python libraries that were used by
  55 +our scripts to upload packages to OBS (Open Build Service). We developed
  56 +a tool called copr-status to keep track of the different stages of the
  57 +lifecycle of each of the individual packages we were working on.
  58 +
... ...
opensym2017/spb.tex
  Diff comments   View file @5588ee4
... ... @@ -151,10 +151,8 @@
151 151 \input{content/05-requirements}
152 152 \input{content/06-architecture}
153 153 \input{content/07-features}
154   -\input{content/08-ux}
155   -\input{content/09-process}
156   -\input{content/10-contributions}
157   -\input{content/11-conclusion}
  154 +\input{content/08-process}
  155 +\input{content/09-conclusion}
158 156  
159 157 %------------------------------------------------------------------------------
160 158 \bibliographystyle{SIGCHI-Reference-Format}
... ...