Compare View

switch
from
...
to
 
Commits (6)
opensym2017/content/04-researchdesign.tex
... ... @@ -27,7 +27,7 @@ managers from the Brazilian Government.
27 27 For the majority of the students, this was a first professional experience.
28 28 Even though, our development process defined a central role on students participation.
29 29  
30   -textbf{Q3:} \textit{How to introduce collaborative and agile
  30 +\textbf{Q3:} \textit{How to introduce collaborative and agile
31 31 practices typical in FLOSS environments in the governmental development process?}
32 32 %
33 33 The software development on Brazilian government is based on a very traditional way,
... ...
opensym2017/content/05-requirements.tex
1 1 \section{Requirements}
2 2 \label{sec:requirements}
3 3  
  4 +\begin{comment}
4 5 In 2013, the SPB Portal had more than 600 thousand unique visitors, generating
5 6 more than 16 million page views with about 50 million hits. By evaluating only
6 7 the main projects, there were more than 15 thousand downloads and 4 thousand
7 8 messages exchanged in their forums. These data illustrates the potential of the
8 9 SPB Portal, even with several limitations in the past.
  10 +\end{comment}
9 11  
10 12 By preparing the evolution project described in this paper, the Brazilian
11 13 government promoted 3 events to collect the requirements, in particular from
... ... @@ -15,32 +17,24 @@ SPB concepts and requirements with IT stakeholders from the Brazilian
15 17 government and public organizations.
16 18  
17 19 After these 3 rounds discussing the new SPB platform, the Brazilian government
18   -listed about 145 requirements and developed a ``mind
19   -map''\footnote{\url{https://softwarepublico.gov.br/social/spb/gallery/mapaconceitual.png}}
20   -to guide the SPB portal evolution. In this scenario, the 10 most voted
21   -requirements were:
  20 +listed about 145 requirements. The 10 most voted requirements were:
  21 +(i) Source code repository with public access;
  22 +(ii) Visit community pages without login;
  23 +(iii) Distributed version control system;
  24 +(iv) Scores of users and developers collaboration;
  25 +(v) Search software by features;
  26 +(vi) Integration with social networks;
  27 +(vii) Repository for future ideas and requirements;
  28 +(viii) Friendly URL to access a public software community page;
  29 +(ix) User feedback about a public software;
  30 +(x) Report of the experience about the use of a public software.
22 31  
23   -\begin{enumerate}
24   -
25   -\item Source code repository with public access;
26   -\item Visit community pages without login;
27   -\item Distributed version control system;
28   -\item Scores of users and developers collaboration;
29   -\item Search software by features;
30   -\item Integration with social networks;
31   -\item Repository for future ideas and requirements;
32   -\item Friendly URL to access a public software community page;
33   -\item User feedback about a public software;
34   -\item Report of the experience about the use of a public software;
35   -
36   -\end{enumerate}
37   -
38   -\begin{figure}[hbt]
39   - \centering
40   - \includegraphics[width=\linewidth]{figures/technological-requirements.png}
41   - \caption{Technological requirements.}
42   - \label{fig:requirements}
43   -\end{figure}
  32 +%\begin{figure}[hbt]
  33 +% \centering
  34 +% \includegraphics[width=\linewidth]{figures/technological-requirements.png}
  35 +% \caption{Technological requirements.}
  36 +% \label{fig:requirements}
  37 +%\end{figure}
44 38  
45 39  
46 40 There were other requirements based on the experience of the IT
... ... @@ -54,23 +48,16 @@ user experience in the new platform.
54 48 At the first moment, we desired to release an initial version that could
55 49 replace the old SPB portal. For that, the first version should have
56 50 features such as:
57   -
58   -\begin{enumerate}
59   -
60   -\item An organized public software catalog;
61   -\item Social network environment (profiles for users, software pages, and community pages);
62   -\item Content Management Systems (CMS) features;
63   -\item Web-based Git repository manager with wiki and issue tracking features;
64   -\item Mailing lists and discussion forums;
65   -
66   -\end{enumerate}
  51 +(i) An organized public software catalog;
  52 +(ii) Social network environment (profiles for users, software pages, and community pages);
  53 +(iii) Content Management Systems (CMS) features;
  54 +(iv) Web-based Git repository manager with wiki and issue tracking features;
  55 +(v) Mailing lists and discussion forums;
67 56  
68 57 Other requirements were also planned during the conception phase of the
69 58 SPB evolution project, such as an integrated search engine and a
70 59 web-based source code static analysis monitor. By analyzing all of these
71   -requirements, we proposed the technological requirements
72   -illustrated in Figure \ref{fig:requirements} to guide the development of
73   -the new SPB platform. In other words, we have designed the SPB evolution
  60 +requirements, we have designed the SPB evolution
74 61 project based on existing FLOSS tools. However, the integration of
75 62 several existing systems that were already implemented in different
76 63 programming languages and frameworks, adding features such as a
... ...
opensym2017/content/06-architecture.tex
... ... @@ -25,46 +25,25 @@ specialized teams for work in the integration process. The teams learned
25 25 how to develop for their assigned systems and contributed to the
26 26 original communities, so that the version we used were not significantly
27 27 different from the original.
28   -
  28 +%
29 29 At the end of the project, the SPB portal was composed of more than ten
30   -systems, such as Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix, and
31   -Munin. The remainder of this section describes the most relevant of
32   -them, as well as how they were integrated into the platform.
33   -
34   -\subsection{Colab}
  30 +systems, such as Colab, Noosfero, Gitlab, and Mezuro.
35 31  
36 32 Colab\footnote{\url{https://github.com/colab}} is a systems integration
37 33 platform for web applications. One of its goals is allowing different
38 34 applications to be combined in such a way that an user does not notice
39 35 the change between applications. For that, Colab provides facilities
40   -for:
41   -
42   -\begin{itemize}
43   - \item \textit{Centralized authentication}: Users are automatically
44   -identified for each application without having to enter login
45   -credentials on each of them;
46   - \item \textit{Visual consistency}: Colab is able to change the HTML
47   -code produced by the integrated applications, such as defining default
48   -template or reusing common HTML pages, in order to provide a unified
49   -interface;
50   - \item \textit{Relaying of events between applications}: The
51   -integrated applications or Colab itself are able to trigger an action
52   -on another application;
53   - \item \textit{Integrated search engine}: It is possible to set up
54   -exactly which data needs to be indexed from each application and how
55   -users are directed to this information on correspondent application;
56   -\end{itemize}
57   -
  36 +for: (i) Centralized authentication, (ii) Visual consistency, (iii) Relaying of events between applications, and (iv) Integrated search engine.
  37 +%
58 38 Colab implements this integration by working as a reverse proxy for the
59 39 integration applications, that is, all external requests pass through
60 40 Colab before reaching them.
61   -
  41 +%
62 42 Initially, Colab had support for a small set of applications (Trac, GNU
63 43 Mailman, and Apache Lucene) and support for them was hard-coded. Our
64 44 team evolved Colab so that it can now receive plugins that add support
65 45 new applications without the need of changes to the Colab core.
66 46  
67   -\subsection{Noosfero}
68 47  
69 48 Noosfero\footnote{\url{http://noosfero.org}} is a software for building
70 49 social and collaboration networks. Besides the classical social
... ... @@ -73,31 +52,29 @@ and a general-purpose CMS (Content Management System). Most of the user
73 52 interactions with SPB is through Noosfero: user registration, project
74 53 home pages and documentation, and contact forms.
75 54  
76   -\subsection{Gitlab}
77 55  
78   -GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository
79   -manager with wiki pages and issue tracking features. Gitlab is a FLOSS
  56 +Gitlab\footnote{\url{http://gitlab.com}} is a web-based Git repository
  57 +manager with wiki pages and issue tracking features. It is a FLOSS
80 58 platform and focuses on delivering a holistic solution that will see
81 59 developers from idea to production seamlessly and on a single platform.
82   -
  60 +%
83 61 Gitlab has several unique features, such as built-in continuous
84 62 integration and continuous deployment, flexible permissions, tracking of
85 63 Work-in-Progress work, moving issues between projects, group-level
86 64 milestones, creating new branches from issues, issues board, and time
87 65 tracking.
88 66  
89   -\subsection{Mezuro}
90 67  
91 68 Mezuro\footnote{\url{http://mezuro.org/}} is a platform to
92 69 collect source code metrics to monitor the internal quality of software written
93 70 in C, C++, Java, Python, Ruby, and PHP.
94   -
  71 +%
95 72 In general, source code metrics tools also do not present a friendly way
96 73 to interpret its results and, even more, do not follow a standardization
97 74 between them. Mezuro collects and presents these results to the end
98 75 user, specially, by analyzing source code metric history during its life
99 76 cycle.
100   -
  77 +%
101 78 The Mezuro platform provides a single interface grouping available
102 79 tools, allows selection and composition of metrics in a flexible manner,
103 80 stores the metrics evolution history, presents results in a friendly
... ... @@ -106,13 +83,6 @@ accordingly to their own context.
106 83  
107 84 \subsection{System unification and User eXperience evolution}
108 85  
109   -\begin{figure}[hbt]
110   - \centering
111   - \includegraphics[width=\linewidth]{figures/arch.png}
112   - \caption{SPB architecture overview.}
113   - \label{fig:architecture}
114   -\end{figure}
115   -
116 86 The conceptual architecture of the platform is presented in Figure
117 87 \ref{fig:architecture}. Colab initially handles all user interaction,
118 88 directing requests to one of the integrated applications. It
... ... @@ -123,6 +93,13 @@ functionality of each application, a search o n the SPB portal might
123 93 return content from any of the applications, be it web pages, mailing
124 94 list posts, or source code.
125 95  
  96 +\begin{figure}[hbt]
  97 + \centering
  98 + \includegraphics[width=.8\linewidth]{figures/arch.png}
  99 + \caption{SPB architecture overview.}
  100 + \label{fig:architecture}
  101 +\end{figure}
  102 +
126 103 The integration of collaborative environments goes beyond functional aspects.
127 104 Offering the population an unified experience across these environments has
128 105 been the key to encourage the use of the platform as it reduces the perception
... ... @@ -148,8 +125,6 @@ interface had to adapt for each individual scenario. In particular
148 125 Noosfero did not yet have a responsive design; we engaged in its
149 126 development and contributed towards that goal.
150 127  
151   -\begin{comment}
152   -%FALTA DE ESPAÇO
153 128 \subsection{Deploy}
154 129  
155 130 The SPB platform was deployed in 7 virtual machines with different functions,
... ... @@ -157,7 +132,7 @@ as we can see in Figure \ref{fig:architecture2}.
157 132  
158 133 \begin{figure*}[hbt]
159 134 \centering
160   - \includegraphics[width=\linewidth]{figures/arch3.png}
  135 + \includegraphics[width=.8\linewidth]{figures/arch3.png}
161 136 \caption{Instanciation view of the SPB architecture.}
162 137 \label{fig:architecture2}
163 138 \end{figure*}
... ... @@ -187,4 +162,4 @@ static analysis tools on source code stored in repository and provide this data
187 162 to Prezento. A social network and CMS (Content Manager System) is provided by
188 163 Noosfero in \textit{social}, and the databases of all tools with a cache
189 164 service are in \textit{database}.
190   -\end{comment}
  165 +
... ...
opensym2017/content/08-process.tex
... ... @@ -2,113 +2,112 @@
2 2 \label{sec:process}
3 3  
4 4 The SPB team was composed of a variety of professionals with different levels
5   -and skills, where most of them were undergraduate students with major in
6   -software engineering (from 4th semester or upper). Since the students could
7   -not dedicate many hours per week to the project, they always had the
  5 +and skills, where most of them were undergraduate students of
  6 +software engineering. Since students could
  7 +not dedicate many hours per week to the project, they had the
8 8 flexibility to negotiate their work schedule during the semester in
9   -order to not harm their classes and coursework. Their daily work routine
  9 +order to not harm their classes and coursework. Their work routine
10 10 in the project included programming and DevOps tasks.
11 11  
12   -The development of SPB project required a vast experience and background that
13   -usually undergraduate students do not have yet. For this reason, a few senior
14   -developers have joined to the project to help with the more difficult issues
  12 +The project required a vast experience and background that
  13 +usually undergraduate students do not have. For this reason, a few senior
  14 +developers have joined the project to help with the more difficult issues
15 15 and to transfer knowledge to the students. Their main task was to provide
16   -solutions for complex problems, in other words, they worked as developers. As
17   -these professionals are very skillful and the project could not fund a full
18   -time work for them, some of them worked partially on the project. In addition,
19   -they lived in a different states spread around Brazil which led much of the
  16 +solutions for complex problems, working as developers. As
  17 +these professionals are very skillful and the project could not fund full-time
  18 +work for all of them, some of them worked partially on the project. In addition,
  19 +they lived in different Brazilian states, which led much of the
20 20 communication to be online.
21 21  
22 22 In short, our work process was based on open and collaborative software
23   -development practices. The development process was defined based on the
24   -adaptation of different agile and FLOSS communities practices, highlighting the
  23 +development practices. The development process was based on the
  24 +adaptation of different agile and FLOSS communities practices, with a
25 25 high degree of automation resulting from DevOps practices. Thus, the work
26 26 process was executed in a cadenced and continuous way.
27 27  
28 28 Finally, the last group of actors of this project was composed of employees
29   -formally working for the Brazilian government, in the Ministry of Planning,
  29 +of the Brazilian Ministry of Planning,
30 30 Development, and Management (MP is the Brazilian acronym). All the project
31   -decisions, validations, and scope definitions were made by them. In this way we
32   -developed software product incrementally, with releases aligned to business
33   -strategic objectives. As one can see, the project had many different
34   -sort of stakeholders that had to be organized and synchronized.
  31 +decisions, validations, and scope definitions were made by them. In this way, we
  32 +incrementally developed a software product with releases aligned to
  33 +strategic business objectives. As one can see, the project had a wide range
  34 +of different stakeholders that had to be organized and synchronized.
35 35  
36 36 \subsection{Team organization}
37 37  
38   -Approximately 70\% of the development teams were composed of software
  38 +Approximately 70\% of the development team was composed of software
39 39 engineering undergraduate students from UnB and they worked physically
40 40 in the same laboratory. Each student had their own schedule based on
41   -their classes, what complicates the implementation of pair programming.
42   -The senior developers tried to synchronize their schedule with the
43   -students on their sub-team. To cope with this environment, we had a few
44   -basic rules which guided the project organization:
  41 +her classes, what complicates the implementation of pair programming.
  42 +The senior developers tried to synchronize their schedule with
  43 +students schedules. To cope with this scenario, we had a few
  44 +basic rules guiding the project organization:
45 45  
46 46 \begin{enumerate}
47 47 \item Classes have high priority for undergraduate students;
48   - \item Work in pairs whenever possible (locally or remotely);
49   - \item There must be one morning or afternoon per week when
50   - \emph{everyone} should be together physically in the laboratory
51   - (except of course the remote team members);
52   - \item Every 3 to 4 months, the senior developers would fly in and work
  48 + \item Pairing whenever possible (locally or remotely);
  49 + \item We had one morning or afternoon per week when
  50 + \emph{everyone}, but the remote members,
  51 + should be together physically in the laboratory;
  52 + \item Every 3 to 4 months the senior developers would travel to work
53 53 alongside the students for a few days.
54 54 \end{enumerate}
55 55  
56 56 With the aforementioned rules we divided all the project into four
57   -different teams: Colab, Noosfero, Design, and DevOps. Each team had one
58   -coach responsible for reducing the communication problem with the other
59   -teams and help the members to organize themselves in the best way for
60   -everyone (always respecting their work time). The coach was one of the
61   -students working in some of the teams with the extra duty of registering
62   -the current tasks developed in the sprint and with the responsibility to
63   -talk with other teams. One important thing to notice is the mutability
64   -of the team and the coach, during the project many students changed
65   -between the teams to try different areas.
  57 +different teams: Colab, Noosfero, Design, and DevOps. One student of each team was the
  58 +coach, responsible for reducing the communication problem with other
  59 +teams and helping the members to organize themselves in the best way for
  60 +everyone (always respecting their work time). The coach
  61 +had also the extra duty of registering
  62 +the current tasks developed in the sprint.
  63 +One important thing to notice is the mutability
  64 +of the team and the coach. During the project many students changed
  65 +their teams to try different areas.
66 66  
67 67 One characteristic of the teams was the presence of (at least) one
68 68 senior per team. This was essential, because hard decisions and complex
69   -problems were usually referred to them. This kept having to take
70   -complicated technical decisions from the coach tole, what encouraged
  69 +problems were usually referred to them. So, it was not the coach role
  70 +to deal with complicated technical decisions, what encouraged
71 71 students to be coaches. Lastly, the senior developers worked directly
72   -with the students, and this was important to give the undergraduate the
73   -opportunity to interact with a savvy professional in his area and to
  72 +with the students, and this was important to give to students the
  73 +opportunity to interact with a savvy professional in their areas and to
74 74 keep the knowledge flowing in the project.
75 75  
76   -Finally, we had to add two last elements of the team organization, that
77   -was essential for the project harmony: the meta-coach and professors.
78   -The former was a software engineer recently graduated and which wanted
79   -to keep working on the project, the latter were professors that
  76 +Finally, we had to add two more elements to the team organization that
  77 +were essential for the project harmony: the meta-coach and professors.
  78 +The former was a software engineer recently graduated that wanted
  79 +to keep working on the project. The latter were professors that
80 80 orchestrated all the interactions between all members of the project.
81 81 The meta-coach usually worked in one specific team and had the extra
82   -task of knowing the current status of all teams. Professors and
83   -meta-coaches worked together to reduce the communication problem between
84   -all the teams. Lastly, all the paperwork tasks, such as reporting on the
85   -project progress to the Ministry, was handled by the professors.
  82 +task of knowing the current status of all teams. Professors and the
  83 +meta-coach worked together to reduce the communication problem among
  84 +teams. Lastly, all the paperwork tasks, such as reporting on the
  85 +project progress to the Ministry, was handled by professors.
86 86  
87 87 \subsection{Communication and management}
88 88  
89   -Our team had many people working together, and most of the seniors worked in a
90   -different city remotely. Also, we tried to keep our work completely clear to
  89 +Our team had many people working together, and most of the seniors worked in
  90 +different cities remotely. Also, we tried to keep our work completely clear to
91 91 the Brazilian government and citizens interested in following the project. To
92   -handle those cases, we used a set of tools to communication and other to manage
93   -the project.
  92 +handle these cases, we used a set of communication and management tools.
94 93  
95   -For communication between members in different places, we used: video
  94 +For communication between members in different places we used: video
96 95 conferencing with shared terminal tools, IRC, and mailing lists. For example,
97 96 when one student had to work in pair with a senior, normally, they used video
98 97 conferencing tool for talking and shared a terminal session (both typing and
99   -seeing the screen in real time). For questions and fast discussion, we used
  98 +seeing each other screen in real time). For questions and fast discussion, we used
100 99 IRC. For general notification, we used the mailing lists.
101 100  
102 101 For managing the project we used the SPB Portal itself; first to validate it by
103 102 ourselves, and also because it had all the required tools. We basically created
104 103 one wiki page per release in the SPB Gitlab instance with a mapping between
105   -strategical, tactical, and operational views. In a practical point of view, one
106   -milestone per user history (feature), and one or more issues for addressing
  104 +strategical, tactical, and operational views. We had one
  105 +milestone per user history (feature) and one or more issues for addressing
107 106 each feature. With this approach we achieved two important goals: keeping all
108   -the management as close possible to to the source code, and tracking every
  107 +the management as close as possible to the source code and tracking every
109 108 feature developed during the project. Our decision to
110   -use Wiki initially was empirical, but later reinforced by a research conducted
111   -by Joseph Chao showing the advantage of using Wikis \cite{chao2007student,
  109 +use the Wiki initially was empirical, but later reinforced by a research conducted
  110 +by Joseph Chao showing the advantage of using Wikis~\cite{chao2007student,
112 111 opensourcestyle}.
113 112  
114 113 \subsection{High-level project management and reporting}
... ... @@ -117,14 +116,14 @@ The Brazilian government used to work with software development in a
117 116 very traditional way. They would frequently focus on documents and not
118 117 on what was, in our opinion, what really matters: working software. This
119 118 dissonance caused us a communication noise with MP, because they would
120   -often question our work style. It was especially hard to convince them
  119 +often question our work style. It was especially hard to convince them
121 120 to accept the idea of open scope and agile development, but after months
122 121 of labor and showing results they stopped resisting.
123 122  
124 123 We defined some level of meeting granularity to avoid generating too
125   -much overhead to the developers. We had a strategical and validating
  124 +much overhead to the developers. We had a strategical and a validating
126 125 meeting with MP (the former once in a month and the latter each 15th
127   -day), release plaining with the entire team (one per month), and finally
  126 +day), a release planning with the entire team (one per month), and finally
128 127 a sprint planning (one each 15th day). Figure \ref{fig:meeting} is a
129 128 diagram that represents our meeting organization.
130 129  
... ... @@ -141,41 +140,42 @@ coach of each team, the meta-coach, and some employees of the MP would
141 140 participate in this meeting. We usually discussed what the team already
142 141 produced since our last meeting, and established the new features for
143 142 the next release. Notice that just part of the team would join this
144   -meeting to avoid generating unnecessary overhead to the developers, but
145   -all the students interested to participate were allowed to join (many
146   -students wanted this experience during the project).
  143 +meeting to avoid generating unnecessary overhead to developers, but
  144 +all the students interested to participate were allowed to join, since many
  145 +students wanted this experience during the project.
147 146  
148 147 After the strategical meeting with Brazilian government agents, we had a
149   -planning turn with all teams together. In this part, each team worked together
  148 +planning phase with all teams together. In this part, each team worked together
150 149 to convert the MP wishes into smaller parts which were represented by the epics of
151   -the release. Each coach was responsible for conducting the planning, and after
152   -that record it on the project wiki (the wiki provided by Gitlab). With this
153   -epic, each 14th day the team have generated their sprint scheduler (with small
  150 +the release. Each coach was responsible for conducting the planning and
  151 +recording it on the project wiki (the wiki provided by Gitlab). With this
  152 +epic, each 14th day the team have documented their sprint schedule (with small
154 153 achievements mapped to issues).
155 154  
156 155 To keep the Brazilian government always updated, we invited them to work
157 156 with us to validate the new features in progress. Normally we had a
158   -meeting each 15th day. Basically, this was our work flow, we always kept
  157 +meeting each 15th day. Basically, this was our work flow. We always kept
159 158 everything extremely open to the MP (our way of working, and the one
160 159 often used by open source projects) and to the team.
161 160  
162   -To keep the track of all of those things we used the SPB itself,
  161 +To keep the track of all of these things we used the SPB itself,
163 162 especially Gitlab. Basically, we had:
164 163  
165 164 \begin{enumerate}
166   - \item Project repository: We have one organization with many repositories;
167   - \item Milestones: Each milestone was used to register a release;
168   - \item Wiki: Each release has one page on wiki with the compilation of
  165 + \item Project repository: we have one organization with many repositories;
  166 + \item Milestones: each milestone was used to register a release;
  167 + \item Wiki: each release has one wiki page with the compilation of the
169 168 strategical meeting;
170   - \item Issues: Each sprint planning generated issues, which we associated to
171   - the specific milestone and updated the wiki with the issue number related
172   - with them. Finally each developer assigned the issue to itself.
  169 + \item Issues: each sprint planning generated issues, which we associated to
  170 + the related milestone and registered on the related wiki page.
  171 + Finally, each developer assigned the issue to herself.
173 172 \end{enumerate}
174 173  
175 174 Notice that this workflow gave us and the Brazilian government agents full
176 175 traceability from a high level view of each feature to the lowest level (code).
177   -It is important to highlight that we converge to this workflow after many
  176 +It is important to highlight that we converged to this workflow after many
178 177 experiments. For example, we used a tool named Redmine for organizing our tasks
179   -during the sprints, however, this tool revealed inefficient for our case since
  178 +during some sprints. However, this tool revealed to be inefficient for our case since
180 179 the government agents lost part of the project traceability. We realized that
181   -centralize all the work in the SPB portal is the best option for our case.
  180 +centralize all the work in the SPB portal was the best option for our case.
  181 +
... ...