Commit f2e42ca6d92c79e79e657736fb974da5d28dfc33

Authored by Paulo Meireles
1 parent 97a0eac5

Re-organizing sections and files

opensym2017/content/01-introduction.tex
@@ -88,3 +88,18 @@ the Brazilian government applying empirical software development methods. This @@ -88,3 +88,18 @@ the Brazilian government applying empirical software development methods. This
88 case can help other projects to overcome similar software engineering challenges 88 case can help other projects to overcome similar software engineering challenges
89 in the future, as well as to illustrate how universities can improve the 89 in the future, as well as to illustrate how universities can improve the
90 real-world experience of their students by means of this kind of project. 90 real-world experience of their students by means of this kind of project.
  91 +
  92 +
  93 +The remainder of this work is organized as follows.
  94 +Section \ref{sec:spb}...
  95 +Section \ref{sec:relatedwork} enumerates a number of related works on the...
  96 +Section \ref{sec:researchdesign} presents the research design...
  97 +Section \ref{sec:requirements} reports ...
  98 +Section \ref{sec:architecture} ...
  99 +Section \ref{sec:features} ...
  100 +Section \ref{sec:ux} ...
  101 +Section \ref{sec:process} ...
  102 +Section \ref{sec:contributions} ...
  103 +Section \ref{sec:lessons} ...
  104 +Finally, Sections \ref{sec:conclusion} concludes the paper highlighting its main contributions and pointing paths to future works.
  105 +
opensym2017/content/03-relatedworks.tex
1 -\section{Related Works and Projects}  
2 -\label{sec:relatedworks} 1 +\section{Related Work}
  2 +\label{sec:relatedwork}
3 3
4 The new SPB platform presented in this paper is a fully integrated 4 The new SPB platform presented in this paper is a fully integrated
5 environment, as we can see in Figure \ref{fig:requirements}, being very 5 environment, as we can see in Figure \ref{fig:requirements}, being very
opensym2017/content/04-requirements.tex
@@ -1,79 +0,0 @@ @@ -1,79 +0,0 @@
1 -\section{Requirements}  
2 -\label{sec:requirements}  
3 -  
4 -In 2013, the SPB Portal had more than 600 thousand unique visitors, generating  
5 -more than 16 million page views with about 50 million hits. By evaluating only  
6 -the main projects, there were more than 15 thousand downloads and 4 thousand  
7 -messages exchanged in their forums. These data illustrates the potential of the  
8 -SPB Portal, even with several limitations in the past.  
9 -  
10 -By preparing the evolution project described in this paper, the Brazilian  
11 -government promoted 3 events to collect the requirements, in particular from  
12 -society point of view: (i) an online form to collect general ideas; (ii) a  
13 -face-to-face meeting with society in general; (iii) a workshop to review the  
14 -SPB concepts and requirements with IT stakeholders from the Brazilian  
15 -government and public organizations.  
16 -  
17 -After these 3 rounds discussing the new SPB platform, the Brazilian government  
18 -listed about 145 requirements and developed a ``mind  
19 -map''\footnote{\url{https://softwarepublico.gov.br/social/spb/gallery/mapaconceitual.png}}  
20 -to guide the SPB portal evolution. In this scenario, the 10 most voted  
21 -requirements were:  
22 -  
23 -\begin{enumerate}  
24 -  
25 -\item Source code repository with public access;  
26 -\item Visit community pages without login;  
27 -\item Distributed version control system;  
28 -\item Scores of users and developers collaboration;  
29 -\item Search software by features;  
30 -\item Integration with social networks;  
31 -\item Repository for future ideas and requirements;  
32 -\item Friendly URL to access a public software community page;  
33 -\item User feedback about a public software;  
34 -\item Report of the experience about the use of a public software;  
35 -  
36 -\end{enumerate}  
37 -  
38 -\begin{figure}[hbt]  
39 - \centering  
40 - \includegraphics[width=\linewidth]{figures/technological-requirements.png}  
41 - \caption{Technological requirements.}  
42 - \label{fig:requirements}  
43 -\end{figure}  
44 -  
45 -  
46 -There were other requirements based on the experience of the IT  
47 -stakeholders from the Brazilian government and from the Brazilian FOSS  
48 -community (that UnB and USP were representing too in this project). The  
49 -new platform would only work properly if there is a unique  
50 -authentication to use the provided tools. Additionally, a unified  
51 -interface was an important non-functional requirement to have a better  
52 -user experience in the new platform.  
53 -  
54 -At the first moment, we desired to release an initial version that could  
55 -replace the old SPB portal. For that, the first version should have  
56 -features such as:  
57 -  
58 -\begin{enumerate}  
59 -  
60 -\item An organized public software catalog;  
61 -\item Social network environment (profiles for users, software pages, and community pages);  
62 -\item Content Management Systems (CMS) features;  
63 -\item Web-based Git repository manager with wiki and issue tracking features;  
64 -\item Mailing lists and discussion forums;  
65 -  
66 -\end{enumerate}  
67 -  
68 -Other requirements were also planned during the conception phase of the  
69 -SPB evolution project, such as an integrated search engine and a  
70 -web-based source code static analysis monitor. By analyzing all of these  
71 -requirements, we proposed the technological requirements  
72 -illustrated in Figure \ref{fig:requirements} to guide the development of  
73 -the new SPB platform. In other words, we have designed the SPB evolution  
74 -project based on existing FOSS tools. However, the integration of  
75 -several existing systems that were already implemented in different  
76 -programming languages and frameworks, adding features such as a  
77 -centralized authentication, unified interface, and a search engine, as  
78 -well as, other back-end features, would require a non-trivial amount of  
79 -work.  
opensym2017/content/04-researchdesign.tex 0 → 100644
@@ -0,0 +1,14 @@ @@ -0,0 +1,14 @@
  1 +\section{Research Design}
  2 +\label{sec:researchdesign}
  3 +
  4 +\begin{description}
  5 +
  6 +\item [RQ1:] \textit{Which strategy could be used to integrate several existing FOSS tools to promote the collaborative software development?}
  7 +
  8 +
  9 +\item [RQ2:] \textit{How to introduce the FOSS collaborative and agile practices to governmental development process?}
  10 +
  11 +
  12 +\item [RQ2:] \textit{How to involve undergraduate students in real-world projects, interacting with real customers from a Government?}
  13 +
  14 +\end{description}
opensym2017/content/05-architecture.tex
@@ -1,171 +0,0 @@ @@ -1,171 +0,0 @@
1 -\section{Architecture}  
2 -\label{sec:architecture}  
3 -  
4 -Based on the extensive list of functional requirements defined by the  
5 -Brazilian Federal Government, we selected some FOSS systems to form our  
6 -solution. We looked for system that together could provide the largest  
7 -subset possible of the requirements, and were fully aware that we would  
8 -need to improve those systems in order to provide the rest. We were also  
9 -convinced that it would be impossible to provide all of those  
10 -requirements with a single tool.  
11 -  
12 -From the point of view of the architecture, two main requirements was  
13 -brought by the Brazilian Federal Government for the new platform:  
14 -  
15 -\begin{enumerate}  
16 - \item \textit{Integrate existing FOSS systems}, with minimal differences from  
17 - their original versions;  
18 - \item \textit{Provide a consistent user interface} across the different  
19 - systems, as well as centralized authentication.  
20 -\end{enumerate}  
21 -  
22 -Adopting existing FOSS systems and minimizing locally-made changes had  
23 -the objective of being able to upgrade to newer versions of the original  
24 -software, benefiting from improvements and maintenance done by the  
25 -existing project communities. Providing a consistent user interface on  
26 -top of those different tools was needed to make the transition between  
27 -the different systems seamless from the point of view of users, which  
28 -would be confused by jumping through completely different interfaces  
29 -while interacting with the portal.  
30 -  
31 -For the first requirement, we identified four main systems that required  
32 -specialized teams for work in the integration process. The teams learned  
33 -how to develop for their assigned systems and contributed to the  
34 -original communities, so that the version we used were not significantly  
35 -different from the original.  
36 -  
37 -At the end of the project, the SPB portal was composed of more than ten  
38 -systems, such as Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix, and  
39 -Munin. The remainder of this section describes the most relevant of  
40 -them, as well as how they were integrated into the platform.  
41 -  
42 -\subsection{Colab}  
43 -  
44 -Colab\footnote{\url{https://github.com/colab}} is a systems integration  
45 -platform for web applications. One of its goals is allowing different  
46 -applications to be combined in such a way that an user does not notice  
47 -the change between applications. For that, Colab provides facilities  
48 -for:  
49 -  
50 -\begin{itemize}  
51 - \item \textit{Centralized authentication}: Users are automatically  
52 -identified for each application without having to enter login  
53 -credentials on each of them;  
54 - \item \textit{Visual consistency}: Colab is able to change the HTML  
55 -code produced by the integrated applications, such as defining default  
56 -template or reusing common HTML pages, in order to provide a unified  
57 -interface;  
58 - \item \textit{Relaying of events between applications}: The  
59 -integrated applications or Colab itself are able to trigger an action  
60 -on another application;  
61 - \item \textit{Integrated search engine}: It is possible to set up  
62 -exactly which data needs to be indexed from each application and how  
63 -users are directed to this information on correspondent application;  
64 -\end{itemize}  
65 -  
66 -Colab implements this integration by working as a reverse proxy for the  
67 -integration applications, that is, all external requests pass through  
68 -Colab before reaching them.  
69 -  
70 -Initially, Colab had support for a small set of applications (Trac, GNU  
71 -Mailman, and Apache Lucene) and support for them was hard-coded. Our  
72 -team evolved Colab so that it can now receive plugins that add support  
73 -new applications without the need of changes to the Colab core.  
74 -  
75 -\subsection{Noosfero}  
76 -  
77 -Noosfero\footnote{\url{http://noosfero.org}} is a software for building  
78 -social and collaboration networks. Besides the classical social  
79 -networking features, it also provides publication features such as blogs  
80 -and a general-purpose CMS (Content Management System). Most of the user  
81 -interactions with SPB is through Noosfero: user registration, project  
82 -home pages and documentation, and contact forms.  
83 -  
84 -\subsection{Gitlab}  
85 -  
86 -GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository  
87 -manager with wiki pages and issue tracking features. Gitlab is a FOSS  
88 -platform and focuses on delivering a holistic solution that will see  
89 -developers from idea to production seamlessly and on a single platform.  
90 -  
91 -Gitlab has several unique features, such as built-in continuous  
92 -integration and continuous deployment, flexible permissions, tracking of  
93 -Work-in-Progress work, moving issues between projects, group-level  
94 -milestones, creating new branches from issues, issues board, and time  
95 -tracking.  
96 -  
97 -\subsection{Mezuro}  
98 -  
99 -Mezuro\footnote{\url{http://mezuro.org/}} \cite{mezuro_oss} is a platform to  
100 -collect source code metrics to monitor the internal quality of software written  
101 -in C, C++, Java, Python, Ruby, and PHP.  
102 -  
103 -In general, source code metrics tools also do not present a friendly way  
104 -to interpret its results and, even more, do not follow a standardization  
105 -between them. Mezuro collects and presents these results to the end  
106 -user, specially, by analyzing source code metric history during its life  
107 -cycle.  
108 -  
109 -The Mezuro platform provides a single interface grouping available  
110 -tools, allows selection and composition of metrics in a flexible manner,  
111 -stores the metrics evolution history, presents results in a friendly  
112 -way, as well as, allows users to customize the given interpretation  
113 -accordingly to their own context.  
114 -  
115 -\subsection{System unification}  
116 -  
117 -\begin{figure}[hbt]  
118 - \centering  
119 - \includegraphics[width=\linewidth]{figures/arch.png}  
120 - \caption{SPB architecture overview.}  
121 - \label{fig:architecture}  
122 -\end{figure}  
123 -  
124 -The conceptual architecture of the platform is presented in Figure  
125 -\ref{fig:architecture}. Colab initially handles all user interaction,  
126 -directing requests to one of the integrated applications. It  
127 -post-processes responses from the applications to apply a consistent  
128 -visual appearance, manages authentication, and provides a unified search  
129 -functionality: instead of using the redundant restricted search  
130 -functionality of each application, a search o n the SPB portal might  
131 -return content from any of the applications, be it web pages, mailing  
132 -list posts, or source code.  
133 -  
134 -% Falar do devops  
135 -\subsection{Deploy}  
136 -  
137 -The SPB platform was deployed in 7 virtual machines with different functions,  
138 -as we can see in Figure \ref{fig:architecture2}.  
139 -  
140 -\begin{figure*}[hbt]  
141 - \centering  
142 - \includegraphics[width=.8\linewidth]{figures/arch3.png}  
143 - \caption{Instanciation view of the SPB architecture.}  
144 - \label{fig:architecture2}  
145 -\end{figure*}  
146 -  
147 -The \textit{reverseproxy} handles the HTTP requests and redirects them to the  
148 -\textit{integration}, the \textit{email} sends and receives e-mails on behalf  
149 -of the platform and the \textit{monitor} keeps the entire environment tracked.  
150 -These three \textit{VMs} mentioned - \textit{reverseproxy}, \textit{email} and  
151 -\textit{monitor} - are accessible via Internet and the other ones are only  
152 -available in the local network created between them.  
153 -  
154 -\textit{Integration} works as a second layer of proxy beneath  
155 -\textit{reverseproxy}, any request to the platform will be handled by it. The  
156 -Colab service provides interface, authentication and search engine integration  
157 -among all the services. When a request is received to a specific service,  
158 -Colab authenticates the user in the target tool, sends the request and makes a  
159 -visual transformation in the HTML page which is the content of the response.  
160 -Another user-oriented feature is the integrated search engine, when the user  
161 -want to find something in the platform Colab will perform the search in the  
162 -whole databases. Colab itself provides a web interface for GNU Mailman and we  
163 -have two others integrated tools in \textit{integration}: Gitlab and Prezento.  
164 -Gitlab provides web interface for Git repositories and issues tracker, and  
165 -Prezento is a front-end for source code static analysis.  
166 -  
167 -The source code static analysis is performed by \textit{mezuro}. It runs some  
168 -static analysis tools on source code stored in repository and provide this data  
169 -to Prezento. A social network and CMS (Content Manager System) is provided by  
170 -Noosfero in \textit{social}, and the databases of all tools with a cache  
171 -service are in \textit{database}.  
opensym2017/content/05-requirements.tex 0 → 100644
@@ -0,0 +1,79 @@ @@ -0,0 +1,79 @@
  1 +\section{Requirements}
  2 +\label{sec:requirements}
  3 +
  4 +In 2013, the SPB Portal had more than 600 thousand unique visitors, generating
  5 +more than 16 million page views with about 50 million hits. By evaluating only
  6 +the main projects, there were more than 15 thousand downloads and 4 thousand
  7 +messages exchanged in their forums. These data illustrates the potential of the
  8 +SPB Portal, even with several limitations in the past.
  9 +
  10 +By preparing the evolution project described in this paper, the Brazilian
  11 +government promoted 3 events to collect the requirements, in particular from
  12 +society point of view: (i) an online form to collect general ideas; (ii) a
  13 +face-to-face meeting with society in general; (iii) a workshop to review the
  14 +SPB concepts and requirements with IT stakeholders from the Brazilian
  15 +government and public organizations.
  16 +
  17 +After these 3 rounds discussing the new SPB platform, the Brazilian government
  18 +listed about 145 requirements and developed a ``mind
  19 +map''\footnote{\url{https://softwarepublico.gov.br/social/spb/gallery/mapaconceitual.png}}
  20 +to guide the SPB portal evolution. In this scenario, the 10 most voted
  21 +requirements were:
  22 +
  23 +\begin{enumerate}
  24 +
  25 +\item Source code repository with public access;
  26 +\item Visit community pages without login;
  27 +\item Distributed version control system;
  28 +\item Scores of users and developers collaboration;
  29 +\item Search software by features;
  30 +\item Integration with social networks;
  31 +\item Repository for future ideas and requirements;
  32 +\item Friendly URL to access a public software community page;
  33 +\item User feedback about a public software;
  34 +\item Report of the experience about the use of a public software;
  35 +
  36 +\end{enumerate}
  37 +
  38 +\begin{figure}[hbt]
  39 + \centering
  40 + \includegraphics[width=\linewidth]{figures/technological-requirements.png}
  41 + \caption{Technological requirements.}
  42 + \label{fig:requirements}
  43 +\end{figure}
  44 +
  45 +
  46 +There were other requirements based on the experience of the IT
  47 +stakeholders from the Brazilian government and from the Brazilian FOSS
  48 +community (that UnB and USP were representing too in this project). The
  49 +new platform would only work properly if there is a unique
  50 +authentication to use the provided tools. Additionally, a unified
  51 +interface was an important non-functional requirement to have a better
  52 +user experience in the new platform.
  53 +
  54 +At the first moment, we desired to release an initial version that could
  55 +replace the old SPB portal. For that, the first version should have
  56 +features such as:
  57 +
  58 +\begin{enumerate}
  59 +
  60 +\item An organized public software catalog;
  61 +\item Social network environment (profiles for users, software pages, and community pages);
  62 +\item Content Management Systems (CMS) features;
  63 +\item Web-based Git repository manager with wiki and issue tracking features;
  64 +\item Mailing lists and discussion forums;
  65 +
  66 +\end{enumerate}
  67 +
  68 +Other requirements were also planned during the conception phase of the
  69 +SPB evolution project, such as an integrated search engine and a
  70 +web-based source code static analysis monitor. By analyzing all of these
  71 +requirements, we proposed the technological requirements
  72 +illustrated in Figure \ref{fig:requirements} to guide the development of
  73 +the new SPB platform. In other words, we have designed the SPB evolution
  74 +project based on existing FOSS tools. However, the integration of
  75 +several existing systems that were already implemented in different
  76 +programming languages and frameworks, adding features such as a
  77 +centralized authentication, unified interface, and a search engine, as
  78 +well as, other back-end features, would require a non-trivial amount of
  79 +work.
opensym2017/content/06-architecture.tex 0 → 100644
@@ -0,0 +1,171 @@ @@ -0,0 +1,171 @@
  1 +\section{Architecture}
  2 +\label{sec:architecture}
  3 +
  4 +Based on the extensive list of functional requirements defined by the
  5 +Brazilian Federal Government, we selected some FOSS systems to form our
  6 +solution. We looked for system that together could provide the largest
  7 +subset possible of the requirements, and were fully aware that we would
  8 +need to improve those systems in order to provide the rest. We were also
  9 +convinced that it would be impossible to provide all of those
  10 +requirements with a single tool.
  11 +
  12 +From the point of view of the architecture, two main requirements was
  13 +brought by the Brazilian Federal Government for the new platform:
  14 +
  15 +\begin{enumerate}
  16 + \item \textit{Integrate existing FOSS systems}, with minimal differences from
  17 + their original versions;
  18 + \item \textit{Provide a consistent user interface} across the different
  19 + systems, as well as centralized authentication.
  20 +\end{enumerate}
  21 +
  22 +Adopting existing FOSS systems and minimizing locally-made changes had
  23 +the objective of being able to upgrade to newer versions of the original
  24 +software, benefiting from improvements and maintenance done by the
  25 +existing project communities. Providing a consistent user interface on
  26 +top of those different tools was needed to make the transition between
  27 +the different systems seamless from the point of view of users, which
  28 +would be confused by jumping through completely different interfaces
  29 +while interacting with the portal.
  30 +
  31 +For the first requirement, we identified four main systems that required
  32 +specialized teams for work in the integration process. The teams learned
  33 +how to develop for their assigned systems and contributed to the
  34 +original communities, so that the version we used were not significantly
  35 +different from the original.
  36 +
  37 +At the end of the project, the SPB portal was composed of more than ten
  38 +systems, such as Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix, and
  39 +Munin. The remainder of this section describes the most relevant of
  40 +them, as well as how they were integrated into the platform.
  41 +
  42 +\subsection{Colab}
  43 +
  44 +Colab\footnote{\url{https://github.com/colab}} is a systems integration
  45 +platform for web applications. One of its goals is allowing different
  46 +applications to be combined in such a way that an user does not notice
  47 +the change between applications. For that, Colab provides facilities
  48 +for:
  49 +
  50 +\begin{itemize}
  51 + \item \textit{Centralized authentication}: Users are automatically
  52 +identified for each application without having to enter login
  53 +credentials on each of them;
  54 + \item \textit{Visual consistency}: Colab is able to change the HTML
  55 +code produced by the integrated applications, such as defining default
  56 +template or reusing common HTML pages, in order to provide a unified
  57 +interface;
  58 + \item \textit{Relaying of events between applications}: The
  59 +integrated applications or Colab itself are able to trigger an action
  60 +on another application;
  61 + \item \textit{Integrated search engine}: It is possible to set up
  62 +exactly which data needs to be indexed from each application and how
  63 +users are directed to this information on correspondent application;
  64 +\end{itemize}
  65 +
  66 +Colab implements this integration by working as a reverse proxy for the
  67 +integration applications, that is, all external requests pass through
  68 +Colab before reaching them.
  69 +
  70 +Initially, Colab had support for a small set of applications (Trac, GNU
  71 +Mailman, and Apache Lucene) and support for them was hard-coded. Our
  72 +team evolved Colab so that it can now receive plugins that add support
  73 +new applications without the need of changes to the Colab core.
  74 +
  75 +\subsection{Noosfero}
  76 +
  77 +Noosfero\footnote{\url{http://noosfero.org}} is a software for building
  78 +social and collaboration networks. Besides the classical social
  79 +networking features, it also provides publication features such as blogs
  80 +and a general-purpose CMS (Content Management System). Most of the user
  81 +interactions with SPB is through Noosfero: user registration, project
  82 +home pages and documentation, and contact forms.
  83 +
  84 +\subsection{Gitlab}
  85 +
  86 +GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository
  87 +manager with wiki pages and issue tracking features. Gitlab is a FOSS
  88 +platform and focuses on delivering a holistic solution that will see
  89 +developers from idea to production seamlessly and on a single platform.
  90 +
  91 +Gitlab has several unique features, such as built-in continuous
  92 +integration and continuous deployment, flexible permissions, tracking of
  93 +Work-in-Progress work, moving issues between projects, group-level
  94 +milestones, creating new branches from issues, issues board, and time
  95 +tracking.
  96 +
  97 +\subsection{Mezuro}
  98 +
  99 +Mezuro\footnote{\url{http://mezuro.org/}} \cite{mezuro_oss} is a platform to
  100 +collect source code metrics to monitor the internal quality of software written
  101 +in C, C++, Java, Python, Ruby, and PHP.
  102 +
  103 +In general, source code metrics tools also do not present a friendly way
  104 +to interpret its results and, even more, do not follow a standardization
  105 +between them. Mezuro collects and presents these results to the end
  106 +user, specially, by analyzing source code metric history during its life
  107 +cycle.
  108 +
  109 +The Mezuro platform provides a single interface grouping available
  110 +tools, allows selection and composition of metrics in a flexible manner,
  111 +stores the metrics evolution history, presents results in a friendly
  112 +way, as well as, allows users to customize the given interpretation
  113 +accordingly to their own context.
  114 +
  115 +\subsection{System unification}
  116 +
  117 +\begin{figure}[hbt]
  118 + \centering
  119 + \includegraphics[width=\linewidth]{figures/arch.png}
  120 + \caption{SPB architecture overview.}
  121 + \label{fig:architecture}
  122 +\end{figure}
  123 +
  124 +The conceptual architecture of the platform is presented in Figure
  125 +\ref{fig:architecture}. Colab initially handles all user interaction,
  126 +directing requests to one of the integrated applications. It
  127 +post-processes responses from the applications to apply a consistent
  128 +visual appearance, manages authentication, and provides a unified search
  129 +functionality: instead of using the redundant restricted search
  130 +functionality of each application, a search o n the SPB portal might
  131 +return content from any of the applications, be it web pages, mailing
  132 +list posts, or source code.
  133 +
  134 +% Falar do devops
  135 +\subsection{Deploy}
  136 +
  137 +The SPB platform was deployed in 7 virtual machines with different functions,
  138 +as we can see in Figure \ref{fig:architecture2}.
  139 +
  140 +\begin{figure*}[hbt]
  141 + \centering
  142 + \includegraphics[width=.8\linewidth]{figures/arch3.png}
  143 + \caption{Instanciation view of the SPB architecture.}
  144 + \label{fig:architecture2}
  145 +\end{figure*}
  146 +
  147 +The \textit{reverseproxy} handles the HTTP requests and redirects them to the
  148 +\textit{integration}, the \textit{email} sends and receives e-mails on behalf
  149 +of the platform and the \textit{monitor} keeps the entire environment tracked.
  150 +These three \textit{VMs} mentioned - \textit{reverseproxy}, \textit{email} and
  151 +\textit{monitor} - are accessible via Internet and the other ones are only
  152 +available in the local network created between them.
  153 +
  154 +\textit{Integration} works as a second layer of proxy beneath
  155 +\textit{reverseproxy}, any request to the platform will be handled by it. The
  156 +Colab service provides interface, authentication and search engine integration
  157 +among all the services. When a request is received to a specific service,
  158 +Colab authenticates the user in the target tool, sends the request and makes a
  159 +visual transformation in the HTML page which is the content of the response.
  160 +Another user-oriented feature is the integrated search engine, when the user
  161 +want to find something in the platform Colab will perform the search in the
  162 +whole databases. Colab itself provides a web interface for GNU Mailman and we
  163 +have two others integrated tools in \textit{integration}: Gitlab and Prezento.
  164 +Gitlab provides web interface for Git repositories and issues tracker, and
  165 +Prezento is a front-end for source code static analysis.
  166 +
  167 +The source code static analysis is performed by \textit{mezuro}. It runs some
  168 +static analysis tools on source code stored in repository and provide this data
  169 +to Prezento. A social network and CMS (Content Manager System) is provided by
  170 +Noosfero in \textit{social}, and the databases of all tools with a cache
  171 +service are in \textit{database}.
opensym2017/content/06-features.tex
@@ -1,76 +0,0 @@ @@ -1,76 +0,0 @@
1 -\section{Features}  
2 -\label{sec:spb}  
3 -  
4 -The new generation of the SPB portal combines features adapted from existing collaborative software  
5 -and features developed by the SPB team. Whenever possible, new functions  
6 -(newly developed or partially modified) were contributed back to the official repositories.  
7 -  
8 -As a result, we have a platform that integrates and harmonizes different features such as  
9 -social networking, mailing list, version control system, content management and  
10 -source code quality monitoring. Our aim was to develop functionalities by reusing  
11 -functionality of collaborative software already integrated to the platform. In  
12 -addition, we tried to keep this integration transparent to end users.  
13 -  
14 -\subsection{Software and Software Community}  
15 -  
16 -In the new SPB portal, each software has a standard set of pages and tools.  
17 -Besides accessing support pages (such as FAQ and installation guide) within the platform, users will be able to download  
18 -different versions of the software and find several mechanisms of software development management.  
19 -  
20 -Focusing on the collaborative development, Mailman was integrated to the platform in order to allow  
21 -the dialogue and communication between developers, users and  
22 -enthusiasts of a determined software. The software has its own mailing list whose privacy  
23 -can be configured/set by administrators.  
24 -  
25 -The software has a social interface area ("software community") where users can find other users, blogs,  
26 -summary of recent activities, or any other relevant community-produced content.  
27 -Users logged to the platform can request membership to different software communities  
28 -and each community member can access and edit restricted content. For this purpose,  
29 -many Noosfero features related to social networking and content management were integrated to the portal.  
30 -  
31 -To assist decision-making, the new SPB has acquired assessment and statistical  
32 -tools. Now, users will be able to rate the software and make comments and all  
33 -information will be avaiable to other users. Moreover, the software has a section  
34 -containing its statistical data, where values are calculated based on data  
35 -provided by users and the system.  
36 -  
37 -The role of the administrator will be present in the software and in its community. The  
38 -administrator is responsible for moderating content, memberships and user  
39 -comments. The administrator is also the one who can make changes in the software homepage  
40 -content.  
41 -  
42 -\subsection{Software Catalog and global search}  
43 -  
44 -The platform also provides a search tool called Software Catalog,  
45 -which allows users to find softwares available in the portal.  
46 -In this catalog, some search options were developed to make the navigation easier,  
47 -such as filters (by type of software or category), sorting and score.  
48 -  
49 -In order to expand the searching scope and cover more types of content, the SPB team  
50 -developed the global search tool. This tool unifies search mechanisms  
51 -provided by the different collaborative software used on the SPB portal. Any user can  
52 -find a public content in the context of social networking, mailing list and  
53 -software development.  
54 -  
55 -\subsection{Software development tools}  
56 -  
57 -Usually, Collaborative Development Environments (CDE) demand a version control  
58 -system, trackers, build tools, knowledge centers, and communication tools  
59 -\cite{collaboration_tools}.  
60 -The new SPB also provides  
61 -tools to encourage developers to keep source code and its  
62 -development activity within the platform. Any created software has, by default, a  
63 -an associated git repository with wiki pages and issue tracking. These tools are  
64 -supplied by the integration of Gitlab into the platform.  
65 -  
66 -Developers can also evaluate the software source code to measure software  
67 -quality. With Mezuro, they can schedule the analysis of the source-code and follow its  
68 -metric results evolution over time. Results of each metric analysis are  
69 -public, which allows greater transparency between the developer and the  
70 -community that uses the software. Thereby, the maintainers can decide if the  
71 -given solution meets the source code quality requirements.  
72 -  
73 -Thus, the SPB became a platform to stimulate the openness of the source code;  
74 -dialogue between users and the development team; and also  
75 -maintenance and evolution of the software, which will provide more  
76 -transparency in government investments.  
opensym2017/content/07-features.tex 0 → 100644
@@ -0,0 +1,76 @@ @@ -0,0 +1,76 @@
  1 +\section{Features}
  2 +\label{sec:features}
  3 +
  4 +The new generation of the SPB portal combines features adapted from existing collaborative software
  5 +and features developed by the SPB team. Whenever possible, new functions
  6 +(newly developed or partially modified) were contributed back to the official repositories.
  7 +
  8 +As a result, we have a platform that integrates and harmonizes different features such as
  9 +social networking, mailing list, version control system, content management and
  10 +source code quality monitoring. Our aim was to develop functionalities by reusing
  11 +functionality of collaborative software already integrated to the platform. In
  12 +addition, we tried to keep this integration transparent to end users.
  13 +
  14 +\subsection{Software and Software Community}
  15 +
  16 +In the new SPB portal, each software has a standard set of pages and tools.
  17 +Besides accessing support pages (such as FAQ and installation guide) within the platform, users will be able to download
  18 +different versions of the software and find several mechanisms of software development management.
  19 +
  20 +Focusing on the collaborative development, Mailman was integrated to the platform in order to allow
  21 +the dialogue and communication between developers, users and
  22 +enthusiasts of a determined software. The software has its own mailing list whose privacy
  23 +can be configured/set by administrators.
  24 +
  25 +The software has a social interface area ("software community") where users can find other users, blogs,
  26 +summary of recent activities, or any other relevant community-produced content.
  27 +Users logged to the platform can request membership to different software communities
  28 +and each community member can access and edit restricted content. For this purpose,
  29 +many Noosfero features related to social networking and content management were integrated to the portal.
  30 +
  31 +To assist decision-making, the new SPB has acquired assessment and statistical
  32 +tools. Now, users will be able to rate the software and make comments and all
  33 +information will be avaiable to other users. Moreover, the software has a section
  34 +containing its statistical data, where values are calculated based on data
  35 +provided by users and the system.
  36 +
  37 +The role of the administrator will be present in the software and in its community. The
  38 +administrator is responsible for moderating content, memberships and user
  39 +comments. The administrator is also the one who can make changes in the software homepage
  40 +content.
  41 +
  42 +\subsection{Software Catalog and global search}
  43 +
  44 +The platform also provides a search tool called Software Catalog,
  45 +which allows users to find softwares available in the portal.
  46 +In this catalog, some search options were developed to make the navigation easier,
  47 +such as filters (by type of software or category), sorting and score.
  48 +
  49 +In order to expand the searching scope and cover more types of content, the SPB team
  50 +developed the global search tool. This tool unifies search mechanisms
  51 +provided by the different collaborative software used on the SPB portal. Any user can
  52 +find a public content in the context of social networking, mailing list and
  53 +software development.
  54 +
  55 +\subsection{Software development tools}
  56 +
  57 +Usually, Collaborative Development Environments (CDE) demand a version control
  58 +system, trackers, build tools, knowledge centers, and communication tools
  59 +\cite{collaboration_tools}.
  60 +The new SPB also provides
  61 +tools to encourage developers to keep source code and its
  62 +development activity within the platform. Any created software has, by default, a
  63 +an associated git repository with wiki pages and issue tracking. These tools are
  64 +supplied by the integration of Gitlab into the platform.
  65 +
  66 +Developers can also evaluate the software source code to measure software
  67 +quality. With Mezuro, they can schedule the analysis of the source-code and follow its
  68 +metric results evolution over time. Results of each metric analysis are
  69 +public, which allows greater transparency between the developer and the
  70 +community that uses the software. Thereby, the maintainers can decide if the
  71 +given solution meets the source code quality requirements.
  72 +
  73 +Thus, the SPB became a platform to stimulate the openness of the source code;
  74 +dialogue between users and the development team; and also
  75 +maintenance and evolution of the software, which will provide more
  76 +transparency in government investments.
opensym2017/content/07-ux.tex
@@ -1,33 +0,0 @@ @@ -1,33 +0,0 @@
1 -\section{User eXperience evolution}  
2 -  
3 -The integration of collaborative environments goes beyond functional aspects.  
4 -Offering the population an unified experience across these environments has  
5 -been the key to encourage the use of the platform as it reduces the perception  
6 -of complexity. Thus, the SPB Portal information architecture was redesigned  
7 -to provide a transparent navigation and to reach users with different profiles.  
8 -A process of harmonization has been employed on the interaction models of each  
9 -tool to reduce the learning curve. At the same time, a new visual style was  
10 -created to unify the navigation experience and to comply with the guidelines of  
11 -the digital communication identity standard established by the Federal  
12 -Government.  
13 -  
14 -With the increase in system features and the addition of new tools, the  
15 -visual style has steadily evolved to keep the navigation unified. Moreover,  
16 -tools from different backgrounds, which in many cases provide similar  
17 -functionality, prompted the development of an unified interface. Some  
18 -features, such as search and user profile editing were eliminated from  
19 -the individual applications, and implemented centrally to ensure a  
20 -consistent look and feel.  
21 -  
22 -Another challenge was responsive web design. The integrated applications  
23 -had varying degrees of support for responsiveness, and the common  
24 -interface had to adapt for each individual scenario. In particular  
25 -Noosfero did not yet have a responsive design; we engaged in its  
26 -development and contributed towards that goal.  
27 -  
28 -After the initial release of the new SPB Portal in 2014, several  
29 -validations activities were implemented in 2015 and 2016. The aim was to  
30 -provide the most wanted features by casual users (such as public  
31 -servants interested in downloads and documentation) immediately, while  
32 -allowing more experienced users (such as developers) to easily drill down  
33 -to the details.  
opensym2017/content/08-process.tex
@@ -1,181 +0,0 @@ @@ -1,181 +0,0 @@
1 -\section{Development Organization and Process}  
2 -\label{sec:process}  
3 -  
4 -The SPB team was composed of a variety of professionals with different levels  
5 -and skills, where most of them were undergraduate students with major in  
6 -software engineering (from 4th semester or upper). Since the students could  
7 -not dedicate many hours per week to the project, they always had the  
8 -flexibility to negotiate their work schedule during the semester in  
9 -order to not harm their classes and coursework. Their daily work routine  
10 -in the project included programming and DevOps tasks.  
11 -  
12 -The development of SPB project required a vast experience and background that  
13 -usually undergraduate students do not have yet. For this reason, a few senior  
14 -developers have joined to the project to help with the more difficult issues  
15 -and to transfer knowledge to the students. Their main task was to provide  
16 -solutions for complex problems, in other words, they worked as developers. As  
17 -these professionals are very skillful and the project could not fund a full  
18 -time work for them, some of them worked partially on the project. In addition,  
19 -they lived in a different states spread around Brazil which led much of the  
20 -communication to be online.  
21 -  
22 -In short, our work process was based on open and collaborative software  
23 -development practices. The development process was defined based on the  
24 -adaptation of different agile and FOSS communities practices, highlighting the  
25 -high degree of automation resulting from DevOps practices. Thus, the work  
26 -process was executed in a cadenced and continuous way.  
27 -  
28 -Finally, the last group of actors of this project was composed of employees  
29 -formally working for the Brazilian government, in the Ministry of Planning,  
30 -Development, and Management (MP is the Brazilian acronym). All the project  
31 -decisions, validations, and scope definitions were made by them. In this way we  
32 -developed software product incrementally, with releases aligned to business  
33 -strategic objectives. As one can see, the project had many different  
34 -sort of stakeholders that had to be organized and synchronized.  
35 -  
36 -\subsection{Team organization}  
37 -  
38 -Approximately 70\% of the development teams were composed of software  
39 -engineering undergraduate students from UnB and they worked physically  
40 -in the same laboratory. Each student had their own schedule based on  
41 -their classes, what complicates the implementation of pair programming.  
42 -The senior developers tried to synchronize their schedule with the  
43 -students on their sub-team. To cope with this environment, we had a few  
44 -basic rules which guided the project organization:  
45 -  
46 -\begin{enumerate}  
47 - \item Classes have high priority for undergraduate students;  
48 - \item Work in pairs whenever possible (locally or remotely);  
49 - \item There must be one morning or afternoon per week when  
50 - \emph{everyone} should be together physically in the laboratory  
51 - (except of course the remote team members);  
52 - \item Every 3 to 4 months, the senior developers would fly in and work  
53 - alongside the students for a few days.  
54 -\end{enumerate}  
55 -  
56 -With the aforementioned rules we divided all the project into four  
57 -different teams: Colab, Noosfero, Design, and DevOps. Each team had one  
58 -coach responsible for reducing the communication problem with the other  
59 -teams and help the members to organize themselves in the best way for  
60 -everyone (always respecting their work time). The coach was one of the  
61 -students working in some of the teams with the extra duty of registering  
62 -the current tasks developed in the sprint and with the responsibility to  
63 -talk with other teams. One important thing to notice is the mutability  
64 -of the team and the coach, during the project many students changed  
65 -between the teams to try different areas.  
66 -  
67 -One characteristic of the teams was the presence of (at least) one  
68 -senior per team. This was essential, because hard decisions and complex  
69 -problems were usually referred to them. This kept having to take  
70 -complicated technical decisions from the coach tole, what encouraged  
71 -students to be coaches. Lastly, the senior developers worked directly  
72 -with the students, and this was important to give the undergraduate the  
73 -opportunity to interact with a savvy professional in his area and to  
74 -keep the knowledge flowing in the project.  
75 -  
76 -Finally, we had to add two last elements of the team organization, that  
77 -was essential for the project harmony: the meta-coach and professors.  
78 -The former was a software engineer recently graduated and which wanted  
79 -to keep working on the project, the latter were professors that  
80 -orchestrated all the interactions between all members of the project.  
81 -The meta-coach usually worked in one specific team and had the extra  
82 -task of knowing the current status of all teams. Professors and  
83 -meta-coaches worked together to reduce the communication problem between  
84 -all the teams. Lastly, all the paperwork tasks, such as reporting on the  
85 -project progress to the Ministry, was handled by the professors.  
86 -  
87 -\subsection{Communication and management}  
88 -  
89 -Our team had many people working together, and most of the seniors worked in a  
90 -different city remotely. Also, we tried to keep our work completely clear to  
91 -the Brazilian government and citizens interested in following the project. To  
92 -handle those cases, we used a set of tools to communication and other to manage  
93 -the project.  
94 -  
95 -For communication between members in different places, we used: video  
96 -conferencing with shared terminal tools, IRC, and mailing lists. For example,  
97 -when one student had to work in pair with a senior, normally, they used video  
98 -conferencing tool for talking and shared a terminal session (both typing and  
99 -seeing the screen in real time). For questions and fast discussion, we used  
100 -IRC. For general notification, we used the mailing lists.  
101 -  
102 -For managing the project we used the SPB Portal itself; first to validate it by  
103 -ourselves, and also because it had all the required tools. We basically created  
104 -one wiki page per release in the SPB Gitlab instance with a mapping between  
105 -strategical, tactical, and operational views. In a practical point of view, one  
106 -milestone per user history (feature), and one or more issues for addressing  
107 -each feature. With this approach we achieved two important goals: keeping all  
108 -the management as close possible to to the source code, and tracking every  
109 -feature developed during the project. Our decision to  
110 -use Wiki initially was empirical, but later reinforced by a research conducted  
111 -by Joseph Chao showing the advantage of using Wikis \cite{chao2007student,  
112 -opensourcestyle}.  
113 -  
114 -\subsection{High-level project management and reporting}  
115 -  
116 -The Brazilian government used to work with software development in a  
117 -very traditional way. They would frequently focus on documents and not  
118 -on what was, in our opinion, what really matters: working software. This  
119 -dissonance caused us a communication noise with MP, because they would  
120 -often question our work style. It was especially hard to convince them  
121 -to accept the idea of open scope and agile development, but after months  
122 -of labor and showing results they stopped resisting.  
123 -  
124 -We defined some level of meeting granularity to avoid generating too  
125 -much overhead to the developers. We had a strategical and validating  
126 -meeting with MP (the former once in a month and the latter each 15th  
127 -day), release plaining with the entire team (one per month), and finally  
128 -a sprint planning (one each 15th day). Figure \ref{fig:meeting} is a  
129 -diagram that represents our meeting organization.  
130 -  
131 -\begin{figure}[hbt]  
132 - \centering  
133 - \includegraphics[width=\linewidth]{figures/meeting_flows.png}  
134 - \caption{Meetings cycles.}  
135 - \label{fig:meeting}  
136 -\end{figure}  
137 -  
138 -In the strategical meeting we usually defined the priorities and new  
139 -features with the Brazilian government. Normally the professors, the  
140 -coach of each team, the meta-coach, and some employees of the MP would  
141 -participate in this meeting. We usually discussed what the team already  
142 -produced since our last meeting, and established the new features for  
143 -the next release. Notice that just part of the team would join this  
144 -meeting to avoid generating unnecessary overhead to the developers, but  
145 -all the students interested to participate were allowed to join (many  
146 -students wanted this experience during the project).  
147 -  
148 -After the strategical meeting with Brazilian government agents, we had a  
149 -planning turn with all teams together. In this part, each team worked together  
150 -to convert the MP wishes into smaller parts which were represented by the epics of  
151 -the release. Each coach was responsible for conducting the planning, and after  
152 -that record it on the project wiki (the wiki provided by Gitlab). With this  
153 -epic, each 14th day the team have generated their sprint scheduler (with small  
154 -achievements mapped to issues).  
155 -  
156 -To keep the Brazilian government always updated, we invited them to work  
157 -with us to validate the new features in progress. Normally we had a  
158 -meeting each 15th day. Basically, this was our work flow, we always kept  
159 -everything extremely open to the MP (our way of working, and the one  
160 -often used by open source projects) and to the team.  
161 -  
162 -To keep the track of all of those things we used the SPB itself,  
163 -especially Gitlab. Basically, we had:  
164 -  
165 -\begin{enumerate}  
166 - \item Project repository: We have one organization with many repositories;  
167 - \item Milestones: Each milestone was used to register a release;  
168 - \item Wiki: Each release has one page on wiki with the compilation of  
169 - strategical meeting;  
170 - \item Issues: Each sprint planning generated issues, which we associated to  
171 - the specific milestone and updated the wiki with the issue number related  
172 - with them. Finally each developer assigned the issue to itself.  
173 -\end{enumerate}  
174 -  
175 -Notice that this workflow gave us and the Brazilian government agents full  
176 -traceability from a high level view of each feature to the lowest level (code).  
177 -It is important to highlight that we converge to this workflow after many  
178 -experiments. For example, we used a tool named Redmine for organizing our tasks  
179 -during the sprints, however, this tool revealed inefficient for our case since  
180 -the government agents lost part of the project traceability. We realized that  
181 -centralize all the work in the SPB portal is the best option for our case.  
opensym2017/content/08-ux.tex 0 → 100644
@@ -0,0 +1,34 @@ @@ -0,0 +1,34 @@
  1 +\section{User eXperience evolution}
  2 +\label{sec:ux}
  3 +
  4 +The integration of collaborative environments goes beyond functional aspects.
  5 +Offering the population an unified experience across these environments has
  6 +been the key to encourage the use of the platform as it reduces the perception
  7 +of complexity. Thus, the SPB Portal information architecture was redesigned
  8 +to provide a transparent navigation and to reach users with different profiles.
  9 +A process of harmonization has been employed on the interaction models of each
  10 +tool to reduce the learning curve. At the same time, a new visual style was
  11 +created to unify the navigation experience and to comply with the guidelines of
  12 +the digital communication identity standard established by the Federal
  13 +Government.
  14 +
  15 +With the increase in system features and the addition of new tools, the
  16 +visual style has steadily evolved to keep the navigation unified. Moreover,
  17 +tools from different backgrounds, which in many cases provide similar
  18 +functionality, prompted the development of an unified interface. Some
  19 +features, such as search and user profile editing were eliminated from
  20 +the individual applications, and implemented centrally to ensure a
  21 +consistent look and feel.
  22 +
  23 +Another challenge was responsive web design. The integrated applications
  24 +had varying degrees of support for responsiveness, and the common
  25 +interface had to adapt for each individual scenario. In particular
  26 +Noosfero did not yet have a responsive design; we engaged in its
  27 +development and contributed towards that goal.
  28 +
  29 +After the initial release of the new SPB Portal in 2014, several
  30 +validations activities were implemented in 2015 and 2016. The aim was to
  31 +provide the most wanted features by casual users (such as public
  32 +servants interested in downloads and documentation) immediately, while
  33 +allowing more experienced users (such as developers) to easily drill down
  34 +to the details.
opensym2017/content/09-contributions.tex
@@ -1,51 +0,0 @@ @@ -1,51 +0,0 @@
1 -\section{Contributions to the upstream communities}  
2 -\label{sec:contributions}  
3 -  
4 -%- projeto feito do jeito certo com relação ao software livre (contribuições upstream etc)  
5 -%* Colab -> RevProxy  
6 -%* Colab, atualização do python/django  
7 -%* Contribuições para o GitLab (autenticação)  
8 -%* Noosfero, atualização do Rails, preparação para federação, nova interface ...  
9 -%* Coper, empacotamentos (obs), omniauth  
10 -  
11 -  
12 -During the execution of this project we made several contributions to  
13 -the upstream communities we interacted with. This occurred due to our  
14 -development process aligned with those of the respective communities.  
15 -During development, we would explicitly discuss the features and bug  
16 -fixes that we were working on with the applicable upstream communities.  
17 -This contributed to improve the developers technical solutions with  
18 -expertise outside of our team, and make it easier for those changes to  
19 -be accepted in the original projects. Having changes accepted upstream  
20 -in turn makes our life easier as it minimizes the delta between our  
21 -codebase and upstream's allowing us to upgrade and benefit from  
22 -development work from others.  
23 -  
24 -In Colab, we helped upstream redesign the entirely architecture,  
25 -enabling the development of plugins to integrate new tools. We also  
26 -added a feature that allowed Colab to run asynchronous tasks, which was  
27 -a major improvement for us since we were developing a complex system. A  
28 -migration to the latest Django version was made (web framework used by  
29 -Colab). Moreover, we worked on RevProxy (the more important dependency  
30 -of Colab) to put it in a good shape, fixing many bugs.  
31 -  
32 -Gitlab was the tool that we made the least number of modifications. We  
33 -contributed with some improvements related with configuration files and  
34 -we developed a new plugin that enables user authentication in Gitlab  
35 -through the REMOTE\_USER HTTP header. This plugin was needed because  
36 -Colab uses this mechanism to manage the authentication.  
37 -  
38 -Noosfero was the tool that contemplated several functional requirements,  
39 -therefore we made a large number of contributions with upstream. We helped to  
40 -migrate to the latest Rails version (web framework used by Noosfero), enable  
41 -the federation implementation (federation with other social networks), and  
42 -decouple the interface and the back-end.  
43 -  
44 -We also made some contributions on the DevOps front. Some members of  
45 -them team became maintainers of some python libraries that were used by  
46 -our scripts to upload packages to OBS (Open Build Service). We developed  
47 -a tool called copr-status to keep track of the different stages of the  
48 -lifecycle of each of the individual packages we were working on.  
49 -  
50 -%TODO: Mezuro  
51 -  
opensym2017/content/09-process.tex 0 → 100644
@@ -0,0 +1,181 @@ @@ -0,0 +1,181 @@
  1 +\section{Development Organization and Process}
  2 +\label{sec:process}
  3 +
  4 +The SPB team was composed of a variety of professionals with different levels
  5 +and skills, where most of them were undergraduate students with major in
  6 +software engineering (from 4th semester or upper). Since the students could
  7 +not dedicate many hours per week to the project, they always had the
  8 +flexibility to negotiate their work schedule during the semester in
  9 +order to not harm their classes and coursework. Their daily work routine
  10 +in the project included programming and DevOps tasks.
  11 +
  12 +The development of SPB project required a vast experience and background that
  13 +usually undergraduate students do not have yet. For this reason, a few senior
  14 +developers have joined to the project to help with the more difficult issues
  15 +and to transfer knowledge to the students. Their main task was to provide
  16 +solutions for complex problems, in other words, they worked as developers. As
  17 +these professionals are very skillful and the project could not fund a full
  18 +time work for them, some of them worked partially on the project. In addition,
  19 +they lived in a different states spread around Brazil which led much of the
  20 +communication to be online.
  21 +
  22 +In short, our work process was based on open and collaborative software
  23 +development practices. The development process was defined based on the
  24 +adaptation of different agile and FOSS communities practices, highlighting the
  25 +high degree of automation resulting from DevOps practices. Thus, the work
  26 +process was executed in a cadenced and continuous way.
  27 +
  28 +Finally, the last group of actors of this project was composed of employees
  29 +formally working for the Brazilian government, in the Ministry of Planning,
  30 +Development, and Management (MP is the Brazilian acronym). All the project
  31 +decisions, validations, and scope definitions were made by them. In this way we
  32 +developed software product incrementally, with releases aligned to business
  33 +strategic objectives. As one can see, the project had many different
  34 +sort of stakeholders that had to be organized and synchronized.
  35 +
  36 +\subsection{Team organization}
  37 +
  38 +Approximately 70\% of the development teams were composed of software
  39 +engineering undergraduate students from UnB and they worked physically
  40 +in the same laboratory. Each student had their own schedule based on
  41 +their classes, what complicates the implementation of pair programming.
  42 +The senior developers tried to synchronize their schedule with the
  43 +students on their sub-team. To cope with this environment, we had a few
  44 +basic rules which guided the project organization:
  45 +
  46 +\begin{enumerate}
  47 + \item Classes have high priority for undergraduate students;
  48 + \item Work in pairs whenever possible (locally or remotely);
  49 + \item There must be one morning or afternoon per week when
  50 + \emph{everyone} should be together physically in the laboratory
  51 + (except of course the remote team members);
  52 + \item Every 3 to 4 months, the senior developers would fly in and work
  53 + alongside the students for a few days.
  54 +\end{enumerate}
  55 +
  56 +With the aforementioned rules we divided all the project into four
  57 +different teams: Colab, Noosfero, Design, and DevOps. Each team had one
  58 +coach responsible for reducing the communication problem with the other
  59 +teams and help the members to organize themselves in the best way for
  60 +everyone (always respecting their work time). The coach was one of the
  61 +students working in some of the teams with the extra duty of registering
  62 +the current tasks developed in the sprint and with the responsibility to
  63 +talk with other teams. One important thing to notice is the mutability
  64 +of the team and the coach, during the project many students changed
  65 +between the teams to try different areas.
  66 +
  67 +One characteristic of the teams was the presence of (at least) one
  68 +senior per team. This was essential, because hard decisions and complex
  69 +problems were usually referred to them. This kept having to take
  70 +complicated technical decisions from the coach tole, what encouraged
  71 +students to be coaches. Lastly, the senior developers worked directly
  72 +with the students, and this was important to give the undergraduate the
  73 +opportunity to interact with a savvy professional in his area and to
  74 +keep the knowledge flowing in the project.
  75 +
  76 +Finally, we had to add two last elements of the team organization, that
  77 +was essential for the project harmony: the meta-coach and professors.
  78 +The former was a software engineer recently graduated and which wanted
  79 +to keep working on the project, the latter were professors that
  80 +orchestrated all the interactions between all members of the project.
  81 +The meta-coach usually worked in one specific team and had the extra
  82 +task of knowing the current status of all teams. Professors and
  83 +meta-coaches worked together to reduce the communication problem between
  84 +all the teams. Lastly, all the paperwork tasks, such as reporting on the
  85 +project progress to the Ministry, was handled by the professors.
  86 +
  87 +\subsection{Communication and management}
  88 +
  89 +Our team had many people working together, and most of the seniors worked in a
  90 +different city remotely. Also, we tried to keep our work completely clear to
  91 +the Brazilian government and citizens interested in following the project. To
  92 +handle those cases, we used a set of tools to communication and other to manage
  93 +the project.
  94 +
  95 +For communication between members in different places, we used: video
  96 +conferencing with shared terminal tools, IRC, and mailing lists. For example,
  97 +when one student had to work in pair with a senior, normally, they used video
  98 +conferencing tool for talking and shared a terminal session (both typing and
  99 +seeing the screen in real time). For questions and fast discussion, we used
  100 +IRC. For general notification, we used the mailing lists.
  101 +
  102 +For managing the project we used the SPB Portal itself; first to validate it by
  103 +ourselves, and also because it had all the required tools. We basically created
  104 +one wiki page per release in the SPB Gitlab instance with a mapping between
  105 +strategical, tactical, and operational views. In a practical point of view, one
  106 +milestone per user history (feature), and one or more issues for addressing
  107 +each feature. With this approach we achieved two important goals: keeping all
  108 +the management as close possible to to the source code, and tracking every
  109 +feature developed during the project. Our decision to
  110 +use Wiki initially was empirical, but later reinforced by a research conducted
  111 +by Joseph Chao showing the advantage of using Wikis \cite{chao2007student,
  112 +opensourcestyle}.
  113 +
  114 +\subsection{High-level project management and reporting}
  115 +
  116 +The Brazilian government used to work with software development in a
  117 +very traditional way. They would frequently focus on documents and not
  118 +on what was, in our opinion, what really matters: working software. This
  119 +dissonance caused us a communication noise with MP, because they would
  120 +often question our work style. It was especially hard to convince them
  121 +to accept the idea of open scope and agile development, but after months
  122 +of labor and showing results they stopped resisting.
  123 +
  124 +We defined some level of meeting granularity to avoid generating too
  125 +much overhead to the developers. We had a strategical and validating
  126 +meeting with MP (the former once in a month and the latter each 15th
  127 +day), release plaining with the entire team (one per month), and finally
  128 +a sprint planning (one each 15th day). Figure \ref{fig:meeting} is a
  129 +diagram that represents our meeting organization.
  130 +
  131 +\begin{figure}[hbt]
  132 + \centering
  133 + \includegraphics[width=\linewidth]{figures/meeting_flows.png}
  134 + \caption{Meetings cycles.}
  135 + \label{fig:meeting}
  136 +\end{figure}
  137 +
  138 +In the strategical meeting we usually defined the priorities and new
  139 +features with the Brazilian government. Normally the professors, the
  140 +coach of each team, the meta-coach, and some employees of the MP would
  141 +participate in this meeting. We usually discussed what the team already
  142 +produced since our last meeting, and established the new features for
  143 +the next release. Notice that just part of the team would join this
  144 +meeting to avoid generating unnecessary overhead to the developers, but
  145 +all the students interested to participate were allowed to join (many
  146 +students wanted this experience during the project).
  147 +
  148 +After the strategical meeting with Brazilian government agents, we had a
  149 +planning turn with all teams together. In this part, each team worked together
  150 +to convert the MP wishes into smaller parts which were represented by the epics of
  151 +the release. Each coach was responsible for conducting the planning, and after
  152 +that record it on the project wiki (the wiki provided by Gitlab). With this
  153 +epic, each 14th day the team have generated their sprint scheduler (with small
  154 +achievements mapped to issues).
  155 +
  156 +To keep the Brazilian government always updated, we invited them to work
  157 +with us to validate the new features in progress. Normally we had a
  158 +meeting each 15th day. Basically, this was our work flow, we always kept
  159 +everything extremely open to the MP (our way of working, and the one
  160 +often used by open source projects) and to the team.
  161 +
  162 +To keep the track of all of those things we used the SPB itself,
  163 +especially Gitlab. Basically, we had:
  164 +
  165 +\begin{enumerate}
  166 + \item Project repository: We have one organization with many repositories;
  167 + \item Milestones: Each milestone was used to register a release;
  168 + \item Wiki: Each release has one page on wiki with the compilation of
  169 + strategical meeting;
  170 + \item Issues: Each sprint planning generated issues, which we associated to
  171 + the specific milestone and updated the wiki with the issue number related
  172 + with them. Finally each developer assigned the issue to itself.
  173 +\end{enumerate}
  174 +
  175 +Notice that this workflow gave us and the Brazilian government agents full
  176 +traceability from a high level view of each feature to the lowest level (code).
  177 +It is important to highlight that we converge to this workflow after many
  178 +experiments. For example, we used a tool named Redmine for organizing our tasks
  179 +during the sprints, however, this tool revealed inefficient for our case since
  180 +the government agents lost part of the project traceability. We realized that
  181 +centralize all the work in the SPB portal is the best option for our case.
opensym2017/content/10-contributions.tex 0 → 100644
@@ -0,0 +1,51 @@ @@ -0,0 +1,51 @@
  1 +\section{Contributions to the upstream communities}
  2 +\label{sec:contributions}
  3 +
  4 +%- projeto feito do jeito certo com relação ao software livre (contribuições upstream etc)
  5 +%* Colab -> RevProxy
  6 +%* Colab, atualização do python/django
  7 +%* Contribuições para o GitLab (autenticação)
  8 +%* Noosfero, atualização do Rails, preparação para federação, nova interface ...
  9 +%* Coper, empacotamentos (obs), omniauth
  10 +
  11 +
  12 +During the execution of this project we made several contributions to
  13 +the upstream communities we interacted with. This occurred due to our
  14 +development process aligned with those of the respective communities.
  15 +During development, we would explicitly discuss the features and bug
  16 +fixes that we were working on with the applicable upstream communities.
  17 +This contributed to improve the developers technical solutions with
  18 +expertise outside of our team, and make it easier for those changes to
  19 +be accepted in the original projects. Having changes accepted upstream
  20 +in turn makes our life easier as it minimizes the delta between our
  21 +codebase and upstream's allowing us to upgrade and benefit from
  22 +development work from others.
  23 +
  24 +In Colab, we helped upstream redesign the entirely architecture,
  25 +enabling the development of plugins to integrate new tools. We also
  26 +added a feature that allowed Colab to run asynchronous tasks, which was
  27 +a major improvement for us since we were developing a complex system. A
  28 +migration to the latest Django version was made (web framework used by
  29 +Colab). Moreover, we worked on RevProxy (the more important dependency
  30 +of Colab) to put it in a good shape, fixing many bugs.
  31 +
  32 +Gitlab was the tool that we made the least number of modifications. We
  33 +contributed with some improvements related with configuration files and
  34 +we developed a new plugin that enables user authentication in Gitlab
  35 +through the REMOTE\_USER HTTP header. This plugin was needed because
  36 +Colab uses this mechanism to manage the authentication.
  37 +
  38 +Noosfero was the tool that contemplated several functional requirements,
  39 +therefore we made a large number of contributions with upstream. We helped to
  40 +migrate to the latest Rails version (web framework used by Noosfero), enable
  41 +the federation implementation (federation with other social networks), and
  42 +decouple the interface and the back-end.
  43 +
  44 +We also made some contributions on the DevOps front. Some members of
  45 +them team became maintainers of some python libraries that were used by
  46 +our scripts to upload packages to OBS (Open Build Service). We developed
  47 +a tool called copr-status to keep track of the different stages of the
  48 +lifecycle of each of the individual packages we were working on.
  49 +
  50 +%TODO: Mezuro
  51 +
opensym2017/content/10-lessons.tex
@@ -1,161 +0,0 @@ @@ -1,161 +0,0 @@
1 -\section{Lessons Learned}  
2 -\label{sec:lessons}  
3 -  
4 -\textbf{How to involve students real-world projects, interacting with  
5 -real customers.}  
6 -%  
7 -Our team was mainly composed of software engineering undergraduate  
8 -student, who had the opportunity to interact with senior developers and  
9 -designers inside the team, as well as with the team of technicians and  
10 -managers from the Brazilian Government, and the management team from  
11 -UnB.  
12 -%  
13 -They interacted with professionals that had diverse expertises, and were  
14 -able to participate in all levels of the software development process.  
15 -This contributed to a great learning opportunity, and for a majority of  
16 -them this was their first professional experience.  
17 -  
18 -\textbf{The participation of experienced professionals is crucial to  
19 -success of the project.}  
20 -One important factor for the students was the composition of the teams  
21 -with the participation of experienced professionals.  
22 -%  
23 -On the technical side, the senior developers and designers would handle  
24 -the more difficult technical decisions, creating a work environment  
25 -where the students could develop their skills in a didactic way without  
26 -pressure.  
27 -%  
28 -On the management side, the active participation of professors -- who  
29 -are, in the end, the ones responsible for the project -- is crucial to  
30 -make sure student participation is conducted in a healthy way, and is an  
31 -instance of leading by example.  
32 -  
33 -\textbf{A balanced relationship between academia and industry.}  
34 -The experience of the SPB project led UnB to develop a work style which  
35 -proved to be appropriate for an educational environment that brings  
36 -academia and industry together.  
37 -%  
38 -The highest priority from the university's point of view is the  
39 -students. Considering this, the activities of the project were never  
40 -prioritized to the detriment of classes and other pedagogical  
41 -activities. In summary, we had students working at different times, part  
42 -time, remotely or locally, always respecting their individual  
43 -conditions, but doing the work in a collective, collaborative and open  
44 -way.  
45 -%  
46 -And even under a potentially adverse environment, the project delivered  
47 -the desired solution with success.  
48 -%  
49 -At the end of the project, we noted that the skills developed by the  
50 -students were at the state of art of the software engineering practice.  
51 -After the project ended, we had team members successfully embracing  
52 -opportunities in public, private, national and international  
53 -organizations, in addition to those students that went into  
54 -entrepreneurship and opened their own companies.  
55 -  
56 -\textbf{Managing different organizational cultures.}  
57 -In the beginning of the project, the Brazilian Government stakeholders  
58 -had certain expectations about the development of project that, let's  
59 -say, didn't exactly match our work based on agile and FOSS practices.  
60 -%  
61 -We had to develop strategies that would support different these  
62 -organizational cultures. As reported in Section \ref{sec:process}, the  
63 -MP is organized in a functional hierarchical organizational structure,  
64 -typically, a traditional development paradigm. Therefore, we had to  
65 -create a translation process between our team and the MP managers who  
66 -managed the project on their side assuming a traditional, waterfall  
67 -process.  
68 -  
69 -\textbf{Manage higher-level and lower-level goals separately.}  
70 -During the initial phase of the project the MP team would often bring  
71 -strategic discussions to technical/operational meetings, where we were  
72 -supposed to discuss practical, technical decisions.  
73 -%  
74 -This produced a highly complex communication and management environment,  
75 -overloading the professors because they were supposed to be responsible  
76 -for maintaining the strategic alignment of the MP synchronized with  
77 -implementation plans of the development team, specially in light of the  
78 -aforementioned culture mismatch. Mixing both concerns in the same  
79 -discussions caused confusion on both sides.  
80 -%  
81 -Towards the middle and end of the project we were able to keep those  
82 -concerns separated, what eased the work of everyone involved.  
83 -  
84 -\textbf{Living with ill-advised political decisions.}  
85 -At the initial phases of the project, by political and personal  
86 -motivation, the main stakeholders from the Brazilian government imposed  
87 -the use of Colab to guide the development of the new SPB platform. Our  
88 -team was totally against the idea because we already knew that Colab was  
89 -a very experimental project and its adoption could dramatically increase  
90 -the project complexity. Even though we provided technical reasons to  
91 -not utilize Colab, MP was adamant and we had to manage this problem. As  
92 -explained in section \ref{sec:architecture} we did massive changes to  
93 -it, and at the end of the project we completely rewrote Colab and made  
94 -it stable. It is important to notice that the MP compelled us to accept  
95 -a technical issue based only on political interests, without considering  
96 -all the resources that would be spent due to that decision. At the end  
97 -of the project, we verified that Colab indeed consumed a vast amount of  
98 -the budget and increased the project complexity. In the end of the  
99 -project and after our analysis on the decision made by the MP, we  
100 -understand that MP's managers are capable of ignoring technical reasons  
101 -in favor to a political decision.  
102 -  
103 -\textbf{Consider sustainability from the beginning.}  
104 -In the process of deploying the SPB platform in the MP structure, we had  
105 -to interact with the MP technicians. We did several workshops, training  
106 -and a meticulous documentation describing all the required procedures to  
107 -update the platform, however, we realized that the MP technicians  
108 -constantly avoid the responsibility. After noticing the aforementioned  
109 -situation, we organized a specific DevOps that completely automated all  
110 -the deployment procedure. We simplified all the platform deployment to a  
111 -few steps: (1) initial configurations (just ssh configuration) and (2)  
112 -the execution of simple commands to completely update the platform. By  
113 -the end of the project, we observed that the MP technicians invariably  
114 -still depended on our support to update the platform even with all the  
115 -automation provided by us. We were sadly left with a feeling of  
116 -uncertainty about the future of the platform after the project ended. In  
117 -hindsight, we realize that the MP dedicated systems analysts and  
118 -managers to the project, but not operations technicians. The later  
119 -should have been more involved with the process so they could at least  
120 -comfortable with managing the platform infrastructure.  
121 -  
122 -% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados  
123 -%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa  
124 -%% afirmação, embora eu e Paulo consigamos perceber isso.  
125 -  
126 -%===========  
127 -% Conclusion  
128 -%===========  
129 -  
130 -\textbf{Final remarks.}  
131 -The portal is available at \url{softwarepublico.gov.br}. All  
132 -documentation, including detailed architecture and operation manuals are  
133 -also available\footnote{\url{https://softwarepublico.gov.br/doc/}  
134 -(in Portuguese only at the moment)}).  
135 -%  
136 -All the integrated tools are FOSS and our contributions were published  
137 -in open repositories, available on the SPB Portal itself. We also  
138 -contributed these features back to the respective communities: that  
139 -benefits those communities, as well as us since we can share future  
140 -development and maintenance effort with other organizations that  
141 -participate in their projects.  
142 -  
143 -  
144 -%* utilização do projeto para formação de recursos humanos (alunos)  
145 -  
146 -%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate  
147 -  
148 -%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)  
149 -  
150 -%* 69 projetos marcados como SPB, de 81 no total na plataforma.  
151 -  
152 -%* 47\% é desenvolvido em PHP.  
153 -  
154 -% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).  
155 -  
156 -% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.  
157 -  
158 -% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.  
159 -  
160 -%* Debater economia de recursos em orgão públicos  
161 -  
opensym2017/content/11-conclusion.tex
@@ -1,35 +0,0 @@ @@ -1,35 +0,0 @@
1 -\section{Conclusion}  
2 -\label{sec:conclusion}  
3 -  
4 -In this paper we present and discuss issues experienced during a government-  
5 -funded project, in partnership with University of Brasilia and University of  
6 -São Paulo, to evolve the Brazilian Public Software portal.  
7 -  
8 -The contributions of this paper are twofold. First, we present how was  
9 -developed an unprecedent platform, delivered to Brazilian government. This  
10 -platform - developed by an heterogenous team of professors, master and  
11 -undergraduate students, IT professionals and governmental managers - provides  
12 -several modern features from the integration of more than 10 FLOSS systems.  
13 -  
14 -Second, the 30 months project which developed this platform results in an  
15 -important case that it is possible to mitigate issues seen as conflicting to IT  
16 -development environment and between industry and academy. We shown that, as  
17 -long as the institution can provide a healthy and challenging environment to  
18 -its students, its is possible to conciliate studies and professional training  
19 -in universities. After the end of the project, some students successfully  
20 -embraced opportunities in public and private sectos, within national borders  
21 -and abroad. Some others went further and started their own companies.  
22 -We also shown that, with some adaptations/"translation processes", was possible  
23 -to conciliate agile methodologies and FOSS practices to develop software to  
24 -governmental organizations with functional hierarchical structures that use  
25 -traditional development paradigm.  
26 -  
27 -In the SPB project it was employed many of our beliefs about FLOSS and Agile  
28 -practices. The team was engaged in creating a friendly environment for everyone  
29 -involved in the project and in showing to government agents another way to  
30 -interact with the FLOSS community and the university. With an open work style,  
31 -the project has seeked to be transparent for the whole society. We belive it is  
32 -still needed to analyze every data produced by the project and its impact on the  
33 -students. For future work, we would conduce a \textit{post-mortem} analyse in the  
34 -project, seeing that we have many open data to be explored and it was applied  
35 -approachs which still need extra validation.  
opensym2017/content/11-lessons.tex 0 → 100644
@@ -0,0 +1,121 @@ @@ -0,0 +1,121 @@
  1 +\section{Lessons Learned}
  2 +\label{sec:lessons}
  3 +
  4 +\textbf{How to involve students real-world projects, interacting with
  5 +real customers.}
  6 +%
  7 +Our team was mainly composed of software engineering undergraduate
  8 +student, who had the opportunity to interact with senior developers and
  9 +designers inside the team, as well as with the team of technicians and
  10 +managers from the Brazilian Government, and the management team from
  11 +UnB.
  12 +%
  13 +They interacted with professionals that had diverse expertises, and were
  14 +able to participate in all levels of the software development process.
  15 +This contributed to a great learning opportunity, and for a majority of
  16 +them this was their first professional experience.
  17 +
  18 +\textbf{The participation of experienced professionals is crucial to
  19 +success of the project.}
  20 +One important factor for the students was the composition of the teams
  21 +with the participation of experienced professionals.
  22 +%
  23 +On the technical side, the senior developers and designers would handle
  24 +the more difficult technical decisions, creating a work environment
  25 +where the students could develop their skills in a didactic way without
  26 +pressure.
  27 +%
  28 +On the management side, the active participation of professors -- who
  29 +are, in the end, the ones responsible for the project -- is crucial to
  30 +make sure student participation is conducted in a healthy way, and is an
  31 +instance of leading by example.
  32 +
  33 +\textbf{A balanced relationship between academia and industry.}
  34 +The experience of the SPB project led UnB to develop a work style which
  35 +proved to be appropriate for an educational environment that brings
  36 +academia and industry together.
  37 +%
  38 +The highest priority from the university's point of view is the
  39 +students. Considering this, the activities of the project were never
  40 +prioritized to the detriment of classes and other pedagogical
  41 +activities. In summary, we had students working at different times, part
  42 +time, remotely or locally, always respecting their individual
  43 +conditions, but doing the work in a collective, collaborative and open
  44 +way.
  45 +%
  46 +And even under a potentially adverse environment, the project delivered
  47 +the desired solution with success.
  48 +%
  49 +At the end of the project, we noted that the skills developed by the
  50 +students were at the state of art of the software engineering practice.
  51 +After the project ended, we had team members successfully embracing
  52 +opportunities in public, private, national and international
  53 +organizations, in addition to those students that went into
  54 +entrepreneurship and opened their own companies.
  55 +
  56 +\textbf{Managing different organizational cultures.}
  57 +In the beginning of the project, the Brazilian Government stakeholders
  58 +had certain expectations about the development of project that, let's
  59 +say, didn't exactly match our work based on agile and FOSS practices.
  60 +%
  61 +We had to develop strategies that would support different these
  62 +organizational cultures. As reported in Section \ref{sec:process}, the
  63 +MP is organized in a functional hierarchical organizational structure,
  64 +typically, a traditional development paradigm. Therefore, we had to
  65 +create a translation process between our team and the MP managers who
  66 +managed the project on their side assuming a traditional, waterfall
  67 +process.
  68 +
  69 +\textbf{Manage higher-level and lower-level goals separately.}
  70 +During the initial phase of the project the MP team would often bring
  71 +strategic discussions to technical/operational meetings, where we were
  72 +supposed to discuss practical, technical decisions.
  73 +%
  74 +This produced a highly complex communication and management environment,
  75 +overloading the professors because they were supposed to be responsible
  76 +for maintaining the strategic alignment of the MP synchronized with
  77 +implementation plans of the development team, specially in light of the
  78 +aforementioned culture mismatch. Mixing both concerns in the same
  79 +discussions caused confusion on both sides.
  80 +%
  81 +Towards the middle and end of the project we were able to keep those
  82 +concerns separated, what eased the work of everyone involved.
  83 +
  84 +\textbf{Living with ill-advised political decisions.}
  85 +At the initial phases of the project, by political and personal
  86 +motivation, the main stakeholders from the Brazilian government imposed
  87 +the use of Colab to guide the development of the new SPB platform. Our
  88 +team was totally against the idea because we already knew that Colab was
  89 +a very experimental project and its adoption could dramatically increase
  90 +the project complexity. Even though we provided technical reasons to
  91 +not utilize Colab, MP was adamant and we had to manage this problem. As
  92 +explained in section \ref{sec:architecture} we did massive changes to
  93 +it, and at the end of the project we completely rewrote Colab and made
  94 +it stable. It is important to notice that the MP compelled us to accept
  95 +a technical issue based only on political interests, without considering
  96 +all the resources that would be spent due to that decision. At the end
  97 +of the project, we verified that Colab indeed consumed a vast amount of
  98 +the budget and increased the project complexity. In the end of the
  99 +project and after our analysis on the decision made by the MP, we
  100 +understand that MP's managers are capable of ignoring technical reasons
  101 +in favor to a political decision.
  102 +
  103 +\textbf{Consider sustainability from the beginning.}
  104 +In the process of deploying the SPB platform in the MP structure, we had
  105 +to interact with the MP technicians. We did several workshops, training
  106 +and a meticulous documentation describing all the required procedures to
  107 +update the platform, however, we realized that the MP technicians
  108 +constantly avoid the responsibility. After noticing the aforementioned
  109 +situation, we organized a specific DevOps that completely automated all
  110 +the deployment procedure. We simplified all the platform deployment to a
  111 +few steps: (1) initial configurations (just ssh configuration) and (2)
  112 +the execution of simple commands to completely update the platform. By
  113 +the end of the project, we observed that the MP technicians invariably
  114 +still depended on our support to update the platform even with all the
  115 +automation provided by us. We were sadly left with a feeling of
  116 +uncertainty about the future of the platform after the project ended. In
  117 +hindsight, we realize that the MP dedicated systems analysts and
  118 +managers to the project, but not operations technicians. The later
  119 +should have been more involved with the process so they could at least
  120 +comfortable with managing the platform infrastructure.
  121 +
opensym2017/content/12-conclusion.tex 0 → 100644
@@ -0,0 +1,75 @@ @@ -0,0 +1,75 @@
  1 +\section{Conclusion}
  2 +\label{sec:conclusion}
  3 +
  4 +In this paper we present and discuss issues experienced during a government-
  5 +funded project, in partnership with University of Brasilia and University of
  6 +São Paulo, to evolve the Brazilian Public Software portal.
  7 +
  8 +The contributions of this paper are twofold. First, we present how was
  9 +developed an unprecedent platform, delivered to Brazilian government. This
  10 +platform - developed by an heterogenous team of professors, master and
  11 +undergraduate students, IT professionals and governmental managers - provides
  12 +several modern features from the integration of more than 10 FLOSS systems.
  13 +
  14 +Second, the 30 months project which developed this platform results in an
  15 +important case that it is possible to mitigate issues seen as conflicting to IT
  16 +development environment and between industry and academy. We shown that, as
  17 +long as the institution can provide a healthy and challenging environment to
  18 +its students, its is possible to conciliate studies and professional training
  19 +in universities. After the end of the project, some students successfully
  20 +embraced opportunities in public and private sectos, within national borders
  21 +and abroad. Some others went further and started their own companies.
  22 +We also shown that, with some adaptations/"translation processes", was possible
  23 +to conciliate agile methodologies and FOSS practices to develop software to
  24 +governmental organizations with functional hierarchical structures that use
  25 +traditional development paradigm.
  26 +
  27 +In the SPB project it was employed many of our beliefs about FLOSS and Agile
  28 +practices. The team was engaged in creating a friendly environment for everyone
  29 +involved in the project and in showing to government agents another way to
  30 +interact with the FLOSS community and the university. With an open work style,
  31 +the project has seeked to be transparent for the whole society. We belive it is
  32 +still needed to analyze every data produced by the project and its impact on the
  33 +students. For future work, we would conduce a \textit{post-mortem} analyse in the
  34 +project, seeing that we have many open data to be explored and it was applied
  35 +approachs which still need extra validation.
  36 +
  37 +The portal is available at \url{softwarepublico.gov.br}. All
  38 +documentation, including detailed architecture and operation manuals are
  39 +also available\footnote{\url{https://softwarepublico.gov.br/doc/}
  40 +(in Portuguese only at the moment)}).
  41 +%
  42 +All the integrated tools are FOSS and our contributions were published
  43 +in open repositories, available on the SPB Portal itself. We also
  44 +contributed these features back to the respective communities: that
  45 +benefits those communities, as well as us since we can share future
  46 +development and maintenance effort with other organizations that
  47 +participate in their projects.
  48 +
  49 +%===========
  50 +% Conclusion
  51 +%===========
  52 +
  53 +% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
  54 +%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
  55 +%% afirmação, embora eu e Paulo consigamos perceber isso.
  56 +
  57 +
  58 +%* utilização do projeto para formação de recursos humanos (alunos)
  59 +
  60 +%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
  61 +
  62 +%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
  63 +
  64 +%* 69 projetos marcados como SPB, de 81 no total na plataforma.
  65 +
  66 +%* 47\% é desenvolvido em PHP.
  67 +
  68 +% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
  69 +
  70 +% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
  71 +
  72 +% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
  73 +
  74 +%* Debater economia de recursos em orgão públicos
  75 +
opensym2017/spb.tex
@@ -171,14 +171,15 @@ @@ -171,14 +171,15 @@
171 \input{content/01-introduction} 171 \input{content/01-introduction}
172 \input{content/02-spb} 172 \input{content/02-spb}
173 \input{content/03-relatedworks} 173 \input{content/03-relatedworks}
174 -\input{content/04-requirements}  
175 -\input{content/05-architecture}  
176 -\input{content/06-features}  
177 -\input{content/07-ux}  
178 -\input{content/08-process}  
179 -\input{content/09-contributions}  
180 -\input{content/10-lessons}  
181 -\input{content/11-conclusion} 174 +\input{content/04-researchdesign}
  175 +\input{content/05-requirements}
  176 +\input{content/06-architecture}
  177 +\input{content/07-features}
  178 +\input{content/08-ux}
  179 +\input{content/09-process}
  180 +\input{content/10-contributions}
  181 +\input{content/11-lessons}
  182 +\input{content/12-conclusion}
182 183
183 %------------------------------------------------------------------------------ 184 %------------------------------------------------------------------------------
184 \bibliographystyle{SIGCHI-Reference-Format} 185 \bibliographystyle{SIGCHI-Reference-Format}