Draft version for SBQS

Paulo Meireles
1 parent 1928b7fa
Showing 14 changed files with 637 additions and 498 deletions Show diff stats
sbqs2017/content/00-abstract.tex
sbqs2017/content/01-introduction.tex
sbqs2017/content/02-spb.tex
sbqs2017/content/03-requirements.tex
sbqs2017/content/04-architecture.tex
sbqs2017/content/05-features.tex
sbqs2017/content/07-process.tex
sbqs2017/content/08-contributions.tex
sbqs2017/content/09-lessons.tex
sbqs2017/figures/arch3.png
sbqs2017/figures/home-SPB_2.png
sbqs2017/figures/technological-requirements.odg
sbqs2017/figures/technological-requirements.png
sbqs2017/spb.tex
@@ -3,7 +3,7 @@
 The Brazilian Public Software (SPB) is a program by the Brazilian Federal
 Government to foster the sharing and collaboration on Free and Open Source
 Software (FOSS) solutions for the public administration. In the one hand,
-Brazilian Public Softwares have some differences from FOSS projects, in
+Brazilian Public Software have some differences from FOSS projects, in
 particular, the software is considered a public good and the Federal government
 assumes some responsibilities related to its use. In the other hand, the
 software development principles are the same: the trend towards
@@ -22,8 +22,27 @@ the platform architecture, features, and the user experience efforts carried
 out, we also discuss our work process, based on agile and free software
 development practices, and the lessons learned in 30 months work on the SPB
 project.
+\end{abstract} 
-\textbf{Keywords:} Brazilian Public Software, Free/Libre/Open Source Software,
-		Software Evolution, Integrated Platform
+\begin{resumo}
+O Software Público Brasileiro (SPB) é um programa do Governo Federal brasileiro
+para promover o compartilhamento e a colaboração de projetos de Software Livre
+para a administração pública. Por um lado, um software público brasileiro tem
+algumas diferenças em relação a um software livre, em especial, o fato do
+software público ser considerado com um bem público and o Governo Federal
+assumir algumas responsabilidades relacionadas ao seu uso. Por outro lado,
+teoricamente, os princípios de desenvolvimento de software são os mesmos:
+tendência a descentralização das decisões sobre o projeto, compartilhamento de
+informações e do desenvolvimento (código), e interação contínua com seus
+usuários.
+%
+Neste contexto, projetamos uma plataforma baseada na evolução e integração de
+ferramentas existentes que promovem a colaboração. Atualmente, o portal SPB
+disponibiliza funcionalidaeds modernas para o desenvolvimento colaborativo de
+software, ajudando a administração pública brasileira a compartilhar suas
+solução.
+%
-\end{abstract}
+
+
+\end{resumo}
 \section{Introduction}
 \label{sec:intro}
-During the last few decades, the Brazilian Federal Government has
-improved its software adoption and development processes. In 2003, the
-recommendation to adopt Free/Open Source Software (FOSS) become a public
-policy. In 2007, the Brazilian Government released a portal called
-Brazilian Public Software (\textit{Software Público Brasileiro} -- SPB,
-in Portuguese), with the goal of sharing FOSS projects developed by, or
-for, the Brazilian Government.
-
-The Brazilian legal instrument on software contracting
-(\textit{Instrução Normativa} 04/2012) mandates that public management
-must consult the SPB Portal to adopt a software solution. The
+During the last few decades, the Brazilian Federal Government has been
+trying to change its software adoption and development processes. For
+instance, in 2003, the recommendation to adopt Free and Open Source
+Software (FOSS) become a public policy. In 2007, the Brazilian
+Government released a portal named Brazilian Public Software
+(\textit{Software Público Brasileiro} -- SPB, in Portuguese), with the
+goal of sharing FOSS projects developed by, or for, the Brazilian
+Government.  Additionally, the Brazilian legal instrument on software
+contracting (known as IN 04/2012) mandates that public agents must give
+priority to solutions available in the SPB Portal.  In short, the
 acquisition of a proprietary solution must be explicitly justified by
-demonstrating that there is no suitable option in the SPB Portal.
-
-Since 2009, however, the SPB Portal was having several technical issues.
-The original codebase was not being developed anymore, and there was a
-large amount of technical debt to overcome. The system was a modified
-version of an existing FOSS platform that was not being developed
-anymore, and the portal maintenance was becoming harder and harder.
-
-From January 2014 to June 2016, a new platform for the SPB Portal was
-designed and developed by the University of Brasília (UnB) and the
-University of São Paulo (USP) in a partnership with the Brazilian
-Ministry of Budget, Planning, and Management(MP). This new Portal was
-designed as an integrated platform for collaborative software
-development. It includes functionality for social networking, mailing
-lists, version control system, and source code quality monitoring. In
-this paper, we present an overview of this new generation of the SPB
-Portal.
-
-The project was developed by a team of 3 professors, 6 professionals, 2
-masters students, and approximately 40 undergraduate students (not all of
-them at the same time, though -- graduations and other events triggered
-changes in the team).
+demonstrating that there is no suitable alternative in the SPB Portal.
+In 2013, the Brazilian Federal Court issued a ruling document
+(\textit{Acórdão 2314/2013}) about an audit survey regarding the use of
+agile methodologies in software development contracts with the public
+administration.
+
+Despite of that, in practice, FOSS or agile methodologies, that is,
+collaborative and empirical software development methods are not widely
+practiced and understood by the Brazilian government agents. Thus, the
+hierarchical and traditional processes from the government and the lack
+of expertise in real-world software development of its agents produces a
+situation of inneficient software development contracts and
+unjustifiable expending of taxpayers' money.
+
+% TODO: ^ references
+
+Since 2009, the SPB Portal was having several technical issues. The
+original codebase was not being developed anymore, and there was a large
+amount of technical debt to overcome. The system was a modified version
+of an existing FOSS platform called
+OpenACS\footnote{\url{http://openacs.org}}, and the old SPB portal was
+not being updated anymore against the official OpenACS releases. In this
+scenario, the portal maintenance was becoming harder and harder.
+
+After some events and meetings to collect requirements from the federal
+government and from the society, a new platform for the SPB Portal was
+developed, among January 2014 and June 2016, by the University of
+Brasília (UnB) and the University of São Paulo (USP) in a partnership
+with the Brazilian Ministry of Budget, Planning, and Management (MP). It
+was designed as an integrated platform for collaborative software
+development., and includes functionality for social networking, mailing
+lists, version control system, and source code quality monitoring. To
+coordinate and develop this project during 30 months, UnB received from
+the Brazilian Federal Government a total of 2,619,965.00 BRL (about
+750,000.00 USD in June 2016).
 \begin{figure*}[hbt]
   \centering
-    \includegraphics[width=.9\linewidth]{figures/home-SPB.png}
+    \includegraphics[width=.9\linewidth]{figures/home-SPB_2.png}
   \caption{The new SPB Portal.}
   \label{fig:spb}
 \end{figure*}
+The project was developed by a team of 3 professors, 2 masters students, and
+approximately 50 undergraduate students (not all of them at the same time,
+though -- graduations and other events triggered changes in the team) together
+with 2 professional designers and 6 senior developers from the FOSS
+communities. The professors and all undergraduate student were from UnB, and
+the master students were from USP. Regarding the designers and senior
+developers, 7 of 8 they were living outside of Brasília: Curitiba/Brazil, São
+Paulo/Brazil, Ribeirão Preto/Brazil, Salvador/Brazil, Punta Cana/Dominican
+Republic, and Montreal/Canada. In other words, we had a team working in
+distributed collaborative virtual environment.
+
 Figure \ref{fig:spb} shows the home page of this integrated platform.
-The development tried to be as faithful as possible to FOSS development.
 All development was done in the open, and the changes we needed in the
-tools were contributed back to their communities.
+FOSS tools were contributed back to their respective communities. Our
+process was based on agile practices and FOSS communities interaction.
+We defined development cycles and released 5 versions of the new SPB
+Portal. The first release (beta) was in September 2014, only 9 months
+from the beginning of the project. The old portal was shut down down in
+September 2015. Finally, the last version illustrated in Figure 1 was
+released in June 2016.
+
+In this paper, we present an overview of this new generation of the SPB
+Portal.  This experience report shares our methodology and process to
+develop this project working with the Brazilian federal government to
+comply with its requirements at the same time to be as faithful as
+possible to FOSS development. Moreover, we discuss several lessons
+learned to provide a distributed collaborative virtual environment
+involving a large undergraduate student team and remote senior
+developers. Lastly, we released an unprecedented platform for the
+Brazilian government applying empirical software development methods.
+This case can help other projects overcome similar software engineering
+challenges in the future, as well as illustrate how universities can
+improve the real-world experience of their students by means of this
+kind of project.
-\section{Free/Open Source Software and Brazilian Public Software}
+\section{Background}
 \label{sec:spb}
 FOSS is a phenomenon that has gained notoriety in recent years and has been
-attarcting the interest of academia. However, since the beginning of computing
+attracting the interest of academia. However, since the beginning of computing
 the majority of developers worked in the way that we now identify as free
-software, that is, sharing code openly. This feature makes the code available
+software, that is, sharing code openly. This openness makes the code available
 for inspection, modification, and use by any person or organization
-\cite{kon2012}, \cite{hippel2003}.
+\cite{hippel2003,kon2012}.
 The elements that distinguish FOSS from other types of software are the
 reasoning about the development process, the economic context, the relationship
 between developers and users, as well as the ethical and legal characteristics
 that relate to the software. In the context of FOSS, user freedom is promoted
-and its development is based on open collaboration and development practices.
-%TODO: Colocar referências sem ser nós mesmo e sem ser em PT-Br
+and its development is based on open collaboration and development practices
+\cite{meirelles2013}.
 From the economic point of view, unlike what happens with proprietary software,
-FOSS promotes the establishment of several suppliers that compete with each
+FOSS promotes the establishment of several suppliers that can compete with each
 other based on the same software. This stronger competition among suppliers
 brings benefits to users because it gives better assurances regarding the
-evolution of the system and induces a reduction in prices. These freedoms and
-assurances on software are guaranteed in Brazil by Law 9610/98 (copyright law).
-Most of the time, this protection from the law complies with the terms
-conferred by a contract related to certain software. This contract is called
-``license''. A software license determines a list of rights that are
+evolution of the system and induces a reduction in prices \cite{kon2012}. These
+freedoms and assurances on software are guaranteed in Brazil by Law 9610/98
+(copyright law).  Most of the time, this protection from the law complies with
+the terms conferred by a contract related to certain software. This contract is
+called ``license''. A software license determines a list of rights that are
 given to, and duties that are imposed on a user of the software. In particular,
 what differentiates FOSS from proprietary software is just the way they are
-licensed\cite{sabino2009}. The FOSS licenses guarantee the right to execute,
+licensed \cite{sabino2009}. The FOSS licenses guarantee the right to execute,
 study, adapt, and improve the software. Example of common FOSS licenses  are
 the \textit{GPL (GNU General Public License)}, the Apache license, the MIT
 license, and the BSD license.
-The SPB portal has been designed in 2005 and released in 2007. In a practical
-view, it is a web system that has consolidated itself as a software project
-sharing environment. It provides a space (community) for each software.
-Therefore, the current platform for SPB was designed to include tools that
-promote collaboration and interaction in communities (by managers, users, and
-developers) of the projects, according to the practices used in FOSS
-communities. This includes e-mail lists, discussion forums, issue trackers,
-version control systems, and social networking environments.
+The original incarnation of SPB portal has been designed in 2005 and
+released in 2007. From a practical point of view, it is a web system
+that has consolidated itself as an environment for sharing software
+projects.  It provides a space (community) for each software.
+Therefore, it was designed to include tools that promote collaboration
+and interaction in communities (by managers, users, and developers) of
+the projects, according to the practices used in FOSS communities. This
+includes mailing lists, discussion forums, issue trackers, version
+control systems, and social networking environments.
 Initially, the purpose of the portal was only to share the software developed
-in the Brazilian government, to reduce the costs of hiring software.  However
+in the Brazilian government, to reduce the costs of hiring software. However,
 it was observed that when softwares were released, their communities were
 formed around those software with several people collaborating and sharing the
 results obtained through the use of those solutions. In this way, some software
 development cooperatives and private companies have shown an interest in making
 their software available on the SPB platform.
-The concept of Brazilian Public Software goes beyond FOSS. In addition to being
-licensed under a FOSS license, a Brazilian Public Software needs to have
-explicit guarantees that it is a public good, and that project must be
-available in the SPB. Being a true public good assumes requirements that can
-not be met solely by means of FOSS licensing. For example, there must be a
-relaxed trademark usage policy by the original vendor that do not stop eventual
-competitors from adversiting services for that same software. Inclusion in the
-SPB also has extra requirements, such as having a public version control
-system, installation manual, hardware requirements specification, etc.
+The concept of Brazilian Public Software goes beyond FOSS. In addition
+to being licensed under a FOSS license, a SPB needs to have explicit
+guarantees that it is a public good, and that project must be available
+in the SPB portal. Being a true public good assumes requirements that
+can not be met solely by means of FOSS licensing. For example, there
+must be a relaxed trademark usage policy by the original vendor that
+does not stop eventual competitors from advertising services for that
+same software. Inclusion in the SPB Portal also has extra requirements,
+such as having a public version control system, installation manual, and
+hardware requirements specification.
-\section{Requirements}
+\section{Requirements and Related Projects}
 \label{sec:requirements}
 In 2013, the SPB Portal had more than 600 thousand unique visitors, generating
 more than 16 million page views with about 50 million hits. By evaluating only
 the main projects, there were more than 15 thousand downloads and 4 thousand
-messages exchanged in their forums. This data illustrates the potential of the
-SPB Portal, even with some limitations in the past.
+messages exchanged in their forums. These data illustrates the potential of the
+SPB Portal, even with several limitations in the past.
 By preparing the evolution project described in this paper, the Brazilian
-government promote 3 events to collect the requirements, in particular from
+government promoted 3 events to collect the requirements, in particular from
 society point of view: (i) an online form to collect general ideas; (ii) a
 face-to-face meeting with society in general; (iii) a workshop to review the
 SPB concepts and requirements with IT stakeholders from the Brazilian
 government and public organizations.
-After these 3 rounds discussing the new SPB platform, the Brazilian government listed
-about 145 requirements and developed a mind
-model\footnote{\url{https://softwarepublico.gov.br/social/spb/gallery/mapaconceitual.png}}
+After these 3 rounds discussing the new SPB platform, the Brazilian government
+listed about 145 requirements and developed a ``mind
+map''\footnote{\url{https://softwarepublico.gov.br/social/spb/gallery/mapaconceitual.png}}
 to guide the SPB portal evolution. In this scenario, the 10 most voted
-requirements are, for example:
+requirements were, for example:
 \begin{enumerate}
@@ -37,26 +37,27 @@ requirements are, for example:
 \begin{figure}[hbt]
   \centering
-    \includegraphics[width=.6\linewidth]{figures/technological-requirements.png}
+    \includegraphics[width=\linewidth]{figures/technological-requirements.png}
   \caption{Technological requirements overview.}
   \label{fig:requirements}
 \end{figure}
-Moreover, there were other requirements based on the experience of the IT
+here were other requirements based on the experience of the IT
 stakeholders from the Brazilian government and from the Brazilian FOSS
-community (that UnB and USP were representing too in this project). The new
-platform just could work properly if there is a unique authentication to use
-the provided tools. Additionally, a unified
-interface was an important non-functional requirement to make easy the user
-experience into the new platform.
+community (that UnB and USP were representing too in this project). The
+new platform would only work properly if there is a unique
+authentication to use the provided tools. Additionally, a unified
+interface was an important non-functional requirement to have a better
+user experience in the new platform.
-At the first moment, we wish to release an initial version to replace the old
-SPB portal. For that, the first version must have some features such as:
+At the first moment, we desired to release an initial version that could
+replace the old SPB portal. For that, the first version should have
+features such as:
 \begin{enumerate}
-\item Organized public software catalog.
+\item An organized public software catalog.
 \item Social network environment (profiles for users, software pages, and community pages).
 \item Content Management Systems (CMS) features.
 \item Web-based Git repository manager with wiki and issue tracking features.
@@ -64,39 +65,42 @@ SPB portal. For that, the first version must have some features such as:
 \end{enumerate}
-Other requirements also were planned during the conception phase of the SPB
-evolution project such as an integrated search engine and a web-based source
-code static analysis monitor. Therefore, by analyzing all of these
-requirements, we propose the technological requirements overview, as
-illustrated in Figure \ref{fig:requirements}, to guide the development of the
-new SPB platform. In other words, we have designed the SPB evolution project
-based on existing FOSS tools. However, the integration of several existing
-systems that already was implemented in different programming language and
-frameworks, adding features such as a unique authentication, a unified
-interface, and a search engine, as well as, other back-end features, is not a
-trivial work.
-
-The new SPB platform is fully an integrated environment, as we can see in
-Figure \ref{fig:requirements}, being very advanced comparing to other related
-projects and initiatives. For example, the USA government has a platform
-designed to improve access to the federal government developed
+Other requirements were also planned during the conception phase of the
+SPB evolution project, such as an integrated search engine and a
+web-based source code static analysis monitor. By analyzing all of these
+requirements, we proposed the technological requirements overview
+illustrated in Figure \ref{fig:requirements} to guide the development of
+the new SPB platform. In other words, we have designed the SPB evolution
+project based on existing FOSS tools. However, the integration of
+several existing systems that were already implemented in different
+programming languages and frameworks, adding features such as a
+centralized authentication, unified interface, and a search engine, as
+well as, other back-end features, would require a non-trivial amount of
+work.
+
+The new SPB platform is a fully integrated environment, as we can see in
+Figure \ref{fig:requirements}, being very advanced in comparison with
+related projects and initiatives. For example, the USA government has a
+platform designed to improve access to the federal government developed
 software\footnote{\url{https://code.gov}}.  Code.gov is an interface to
-organize the USA government projects and, in short, make easy that their users
-and developers obtain some information and access their source code
+organize the USA government projects and, in short, make it easy for
+users and developers to obtain information and access their source code
 repositories at GitHub. However, there are not social networking and CMS
-features, as well as, other communication resources provided by that platform.
+features, as well as, other communication resources provided by that
+platform.
-Also, there are two initiatives in Europe:
+Additionally, there are two initiatives in Europe:
 OSOR\footnote{\url{https://joinup.ec.europa.eu/community/osor}} and
-OW2\footnote{\url{http://ow2.org}}. The Open Source Observatory (OSOR) is a
-community hosted in the JoinUp platform powereded by the European Commission.
-OSOR aims exchanging information, experiences and best practices around FOSS
-solutions for use in public administrations. Summarily, it helps to find a FOSS
-made available by other public administrations, providing access to information
-such as news, events, studies and solutions related to implementation of open
-source software. It also offers forum discussions and community mailing lists,
-but it does not have an integrated source code repository manager and for the
-each project has a link to its own external repository (or its tarball file).
+OW2\footnote{\url{http://ow2.org}}. The Open Source Observatory (OSOR)
+is a community hosted in the JoinUp platform powered by the European
+Commission.  OSOR aims at exchanging information, experiences and best
+practices around the use of FOSS in the public administration. It helps
+to find a FOSS made available by other public administrations, providing
+access to information such as news, events, studies and solutions
+related to implementation of open source software. It also offers forum
+discussions and community mailing lists, but it does not have an
+integrated source code repository manager and for the each project there
+is a link to its own external repository (or its tarball file).
 %
 OW2 is a FOSS community to promote the development of FOSS middleware, generic
 business applications, cloud computing platforms and foster a community and
@@ -104,33 +108,31 @@ business ecosystem. In short, it aims to support the development, deployment
 and management of distributed applications with a focus on FOSS middleware and
 related development and management tools.
 %
-Moreover, from the European Commission in 2007 until 20011, there were the
-QualiPSo project that aims to provide to FOSS users, developers, and consumers,
-quality resources and expertise on the various topics related to FOSS. The
+Moreover, from the European Commission in 2007 until 20011, there was the
+QualiPSo project that aimed at providing FOSS users, developers, and consumers,
+with quality resources and expertise on the various topics related to FOSS. The
 QualiPSo project also had planned to develop a platform called QualiPSo
 Factory but it was not fully completed.
-In Latin American has an initiative based on the SPB project called Software
-Publico Regional\footnote{\url{http://softwarepublicoregionalbeta.net}}. From
-the practical point of view, it provides a customized Gitlab instance to share
+In Latin American there is an initiative based on the SPB project called ``Software
+Publico Regional''\footnote{\url{http://softwarepublicoregionalbeta.net}}. From
+a practical point of view, it provides a customized Gitlab instance to share
 the source code and documentation of the project from the involved countries.
 %
-Such as Brazil, Chile has its own portal also called Software
-Publico\footnote{\url{http://www.softwarepublico.gob.cl}}. The user can create
-content in the communities (news items, documents, wiki pages), but all
-repository is available at the Bitbucket
+Like Brazil, Chile has its own portal also called ``Software
+Publico''\footnote{\url{http://www.softwarepublico.gob.cl}}. Users can create
+content in the communities (news items, documents, wiki pages), but
+source code repositories are available at the Bitbucket
 platform\footnote{\url{https://bitbucket.org/softwarepublico}}.
-The Brazilian government needed to evolute the SPB project that exists since
-2005. In 2013, when we started this project, the SPB Portal had about 200
-thousand registered users. We could not just contact these users and ask them
-to register an account at Github as well. Moreover, after the Edward Snowden
-case, the Brazilian government approved a specific law decree (8.135/2013) to
-rule its communication service. Summarily, services like Gmail, Google Drive,
-Dropbox, Live, Outlook, iCloud, as well, Google Groups, Github, etc do not
-could be used by a Brazilian public agent as tool for your work. To use these
-kinds of services the Brazilian government needs to provide them to itself.
-Thus, we developed our own solution to cover all the requirements, producing a
-complete governmental integrated platform for collaborative software
-development.
-
+The Brazilian government needed to evolve the SPB project that
+existedince 2005. In 2013, when we started this project, the SPB Portal
+had about 200 thousand registered users. We could not just contact these
+users and ask them to register an account at Github as well. Moreover,
+after the Edward Snowden case, the Brazilian government approved a
+specific law decree (8.135/2013) to rule its communication services,
+requiring the public administration to host its information systems to
+be provided by itself, what rules out usage of private platforms,
+specially ones provided by foreign companies. We thus developed our own
+solution to cover all the requirements, producing a complete
+governmental integrated platform for collaborative software development.
 \section{Architecture}
 \label{sec:architecture}
-Based on the great list of functional requirements desired by Brazilian Federal
-Government we decided to select some FOSS systems that already contemplate some
-of them and improve these systems. And bringing the idea of community, it is
-undoable build a platform to be used by communities, which is a complex
-scenario, using just one tool.
+Based on the extensive list of functional requirements defined by the
+Brazilian Federal Government, we selected some FOSS systems to form our
+solution. We looked for system that together could provide the largest
+subset possible of the requirements, and were fully aware that we would
+need to improve those systems in order to provide the rest. We were also
+convinced that it would be impossible to provide all of those
+requirements with a single tool.
-At the point of view of the architecture, two main requirements was brought by
-the Brazilian Federal Government for the new platform:
+From the point of view of the architecture, two main requirements was
+brought by the Brazilian Federal Government for the new platform:
 \begin{enumerate}
   \item \textit{Integrate existing FOSS systems}, with minimal differences from
@@ -17,69 +19,81 @@ the Brazilian Federal Government for the new platform:
     systems, as well as centralized authentication.
 \end{enumerate}
-Adopting existing FOSS systems by the government could bring the benefit of
-improvements done by the upstream communities, and the maintenance effort. On
-the other hand, integrating different tools with distinct intent, it is not an
-easy task and it was important to have a consistent user interface which
-justifies the last requirement.
+Adopting existing FOSS systems and minimizing locally-made changes had
+the objective of being able to upgrade to newer versions of the original
+software, benefiting from improvements and maintenance done by the
+existing project communities. Providing a consistent user interface on
+top of those different tools was needed to make the transition between
+the different systems seamless from the point of view of users, which
+would be confused by jumping through completely different interfaces
+while interacting with the portal.
 For the first requirement, we identified four main systems that required
-specialized teams for work in the integration process. The teams learned how to
-develop for their assigned systems and contributed to the  original
-communities, so that the version we used were not significantly different from
-the original.
+specialized teams for work in the integration process. The teams learned
+how to develop for their assigned systems and contributed to the
+original communities, so that the version we used were not significantly
+different from the original.
-At the end of the project, SPB portal was composed of more than ten systems
-among them we can highlight: Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix,
-Munin, and so forth. The following sections explained a little bit of Colab,
-Noosfero, Mezuro, and Gitlab (the main tools which we contributed). Below, we
-described how we integrated those tools and conclude with the deployment.
+At the end of the project, the SPB portal was composed of more than ten
+systems, such as Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix, and
+Munin. The remainder of this section describes the most relevant of
+them, as well as how they were integrated into the platform.
 \subsection{Colab}
-Colab integrates web applications as its main goal, the user of this composed
-system should not notice the change between the integrated applications. Colab
-was designed to use one plugin for each system under its domain, this is
-guaranteed by four levels of integration:
+Colab\footnote{\url{https://github.com/colab}} is a systems integration
+platform for web applications. One of its goals is allowing different
+applications to be combined in such a way that an user does not notice
+the change between applications. For that, Colab provides facilities
+for:
 \begin{itemize}
-  \item Authentication
-  \item Visual
-  \item Events
-  \item Data and search engine
+  \item Centralized authentication
+  \item Visual consistency
+  \item Relaying of events between applications
+  \item Integrated search engine
 \end{itemize}
-The aforementioned integrations levels were possible, because Colab works as a
-reverse proxy, therefore all external requests pass through it.
-
-Single Sign-On (SSO) is used to login users throughout all integrated
-applications. REMOTE\_USER HTTP header is sent to the applications and we
-expect that they know how to handle it, since this is a common authentication
-mechanism. The integrated applications should be on a local network to avoid
-security issues. With this the user will be able to navigate through the
-platform applications and will not be asked for authentication credentials.
-
-As Colab\footnote{\url{https://github.com/colab}} is a reverse proxy, it makes
-some HTML transformations in the HTTP response of integrated applications to
-provide a unified interface. Allows one to define some default templates to be
-used by all applications and overwrite when needed in its plugin. This approach
-allowed us to reuse some HTML pages which facilitates maintenance.
-
-A publish-subscribe implementation was used to handle events in the platform.
-Thus, if some application want to trigger some action in case that other
-application do something is possible. A registration of the desired events and
-implementation of the handlers must be done in the plugin of each application.
-This bring us many options to innovate in these integrations.
-
-A integrated search engine is provided by Colab. Each plugin specify which data
-will be indexed and will became available for the users, just need an simple
-implementation of how should perform the collection of data.
+Colab implements this integration by working as a reverse proxy for the
+integration applications, that is, all external requests pass through
+Colab before reaching them.
+
+Centralized authentication is handled by letting users authenticate to
+Colab, which then sends a REMOTE\_USER HTTP header to applications,
+which are expected to be pre-configured to accept that as an indication
+of the current user (REMOTE\_USER is a standard authentication mechanism
+for web applications). This allows users to be automatically identified
+to each of the applications without having to enter credentials for each
+of them.
+%
+Of course, this requires that the integrated applications are not
+accessible on the public internet, otherwise malicious users could send
+their own crafted REMOTE\_USER headers and impersonate any user.
+
+For the visual consistency, Colab is able to make transformations to the
+HTML produced by the integrated applications, ir order to provide a
+unified interface. It allows one to define default templates to be used
+by all applications, as well as providing extra ones in a plugin. This
+approach allowed us to reuse common HTML pages, what facilitates
+maintenance.
+
+Colab also provides a publish-subscribe event system. This allows one of
+the applications, or Colab itself, to trigger an action in another
+application. For example, when a user registers with Colab, all
+applications need to be notified in order to create their own internal
+representation of that user account.
+
+Colab also provides an integrated search engine, which can be configured
+to specify exactly what data needs to be indexed for each of the
+applications, and how to direct the user to that piece of information on
+the specific application.
 Initially, Colab had support for a small set of applications (Trac, GNU
-Mailman, and Apache Lucene) and all of them was hard-coded. Our team evolved
-Colab so that it can now receive plugins to add support for new applications
-with minimal changes to its existing core. We developed plugins to be used in
-the SPB platform: Noosfero, GitLab, and Mezuro.
+Mailman, and Apache Lucene) and support for them was hard-coded. Our
+team evolved Colab so that it can now receive plugins that add support
+new applications without the need of changes to the Colab core. We also
+developed plugins to be used in the SPB platform: Noosfero, GitLab, and
+Mezuro.
 \subsection{Noosfero}
@@ -92,44 +106,40 @@ home pages and documentation, and contact forms.
 \subsection{Gitlab}
-GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository manager
-with wiki pages and issue tracking features. On the one hand, Github is the
-most famous and used Git repository manager. On the other hand, it is not a
-FOSS. Gitlab is a FOSS platform and focuses on delivering a holistic solution
-that will see developers from idea to production seamlessly and on a single
-platform. Thus, we have worked on it to integrated to Colab.
-
-Moreover, Gitlab has several features working properly than Github that is
-interesting in SPB context such as free for private projects, built-in
-continuous integration and continuous deployment with best practices, more
-control during downtime and upgrade, flexible permissions, Work-in-Progress
-protection, move issues between projects, Group-level milestones, Create new
-branches from issues, Issue board, Time tracking, as well new features and
-improvements every month on the 22nd.
+GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository
+manager with wiki pages and issue tracking features. Gitlab is a FOSS
+platform and focuses on delivering a holistic solution that will see
+developers from idea to production seamlessly and on a single platform.
+
+Gitlab has several unique features, such as built-in continuous
+integration and continuous deployment, flexible permissions, tracking of
+Work-in-Progress work, moving issues between projects, group-level
+milestones, creating new branches from issues, issues board, and time
+tracking.
 \subsection{Mezuro}
 Mezuro\footnote{\url{http://mezuro.org/}} is a platform to collect source code
-metric to monitor the internal quality of softwares written in C, C++,
-Java, Python, Ruby, and PHP. GNU Mailman is used for mailing lists.
+metrics to monitor the internal quality of software written in C, C++,
+Java, Python, Ruby, and PHP.
-In general, source code metric tools also do not present a friendly way to
-interpret its results and, even more, do not follow a standardization between
-them. Mezuro brings and presents these results to
-the end user, specially, by analyzing source code metric history during its
-life cycle.
+In general, source code metrics tools also do not present a friendly way
+to interpret its results and, even more, do not follow a standardization
+between them. Mezuro collects and presents these results to the end
+user, specially, by analyzing source code metric history during its life
+cycle.
-The Mezuro platform provides a single interface
-grouping available tools, allows selection and composition of metrics in a
-flexible manner, stores the metrics evolution history, presents
-results in a friendly way, as well as, allows users to customize the given
-interpretation accordingly to their own context.
+The Mezuro platform provides a single interface grouping available
+tools, allows selection and composition of metrics in a flexible manner,
+stores the metrics evolution history, presents results in a friendly
+way, as well as, allows users to customize the given interpretation
+accordingly to their own context.
 \subsection{System unification}
 \begin{figure}[hbt]
   \centering
-    \includegraphics[width=.5\linewidth]{figures/arch.png}
+    \includegraphics[width=\linewidth]{figures/arch.png}
   \caption{SPB architecture overview.}
   \label{fig:architecture}
 \end{figure}
@@ -143,42 +153,5 @@ functionality: instead of using the redundant restricted search
 functionality of each application, a search in the SPB portal might
 return content from any of the applications, be it web pages, mailing
 list posts, or source code.
-
-% Falar do devops
-\subsection{Deploy}
-
-The SPB platform was deployed in 7 virtual machines with different functions,
-as we can see in Figure \ref{fig:architecture2}.
-
-\begin{figure*}[hbt]
-  \centering
-    \includegraphics[width=.8\linewidth]{figures/arch2.png}
-  \caption{Instanciation view of the SPB architecture.}
-  \label{fig:architecture2}
-\end{figure*}
-
-The \textit{reverseproxy} handles the HTTP requests and redirects them to the
-\textit{integration}, the \textit{email} sends and receives e-mails on behalf
-of the platform and the \textit{monitor} keeps the entire environment tracked.
-These three \textit{VMs} mentioned - \textit{reverseproxy}, \textit{email} and
-\textit{monitor} - are accessible via Internet and the other ones are only
-available in the local network created between them.
-
-\textit{Integration} works as a second layer of proxy beneath
-\textit{reverseproxy}, any request to the platform will be handled by it. The
-Colab service provides interface, authentication and search engine integration
-among all the services. When a request is received to a specific service,
-Colab authenticates the user in the target tool, sends the request and makes a
-visual transformation in the HTML page which is the content of the response.
-Another user-oriented feature is the integrated search engine, when the user
-want to find something in the platform Colab will perform the search in the
-whole databases. Colab itself provides a web interface for GNU Mailman and we
-have two others integrated tools in \textit{integration}: Gitlab and Prezento.
-Gitlab provides web interface for Git repositories and issues tracker, and
-Prezento is a front-end for source code static analysis.
-
-The source code static analysis is performed by \textit{mezuro}. It runs some
-static analysis tools on source code stored in repository and provide this data
-to Prezento. A social network and CMS (Content Manager System) is provided by
-Noosfero in \textit{social}, and the databases of all tools with a cache
-service are in \textit{database}.
+%
+The SPB platform was deployed in 7 virtual machines with different functions.
 \section{Features}
 \label{sec:spb}
-The new generation of the SPB portal combines adapted features of existing collaborative softwares
+The new generation of the SPB portal combines features adapted from existing collaborative software
 and features developed by the SPB team. Whenever possible, new functions 
-(newly developed or partially modified) were sent to official repositories, as a contribution.
+(newly developed or partially modified) were contributed back to the official repositories.
 As a result, we have a platform that integrates and harmonizes different features such as
 social networking, mailing list, version control system, content management and
 source code quality monitoring. Our aim was to develop functionalities by reusing
-functions of collaborative softwares already integrated to the platform. In
+functionality of collaborative software already integrated to the platform. In
 addition, we tried to keep this integration transparent to end users.
 \subsection{Software and Software Community}
@@ -17,47 +17,47 @@ In the new SPB portal, each software has a standard set of pages and tools.
 Besides accessing support pages (such as FAQ and installation guide) within the platform, users will be able to download
 different versions of the software and find several mechanisms of software development management.
-Focusing on the collaborative development, the Mailman was integrated to the platform in order to allow
+Focusing on the collaborative development, Mailman was integrated to the platform in order to allow
 the dialogue and communication between developers, users and
 enthusiasts of a determined software. The software has its own mailing list whose privacy
 can be configured/set by administrators. 
-The software has a social interface area (aka "software community") where users can find other users, blogs,
+The software has a social interface area ("software community") where users can find other users, blogs,
 summary of recent activities, or any other relevant community-produced content. 
 Users logged to the platform can request membership to different software communities 
 and each community member can access and edit restricted content. For this purpose, 
-many Noosfero features related to social network and content management was integrated to the portal.
+many Noosfero features related to social networking and content management were integrated to the portal.
 To assist decision-making, the new SPB has acquired assessment and statistical
 tools. Now, users will be able to rate the software and make comments and all
 information will be avaiable to other users. Moreover, the software has a section
-containing  its statistical data, where values are calculated through data
+containing  its statistical data, where values are calculated based on data
 provided by users and the system.
 The role of the administrator will be present in the software and in its community. The
 administrator is responsible for moderating content, memberships and user
-comments. He is also the one who can make changes in the software homepage
+comments. The administrator is also the one who can make changes in the software homepage
 content.
 \subsection{Software Catalog and global search}
 The platform also provides a search tool called Software Catalog, 
-which allows users to find softwares available in its directory.
+which allows users to find softwares available in the portal.
 In this catalog, some search options were developed to make the navigation easier,
 such as filters (by type of software or category), sorting and score.
 In order to expand the searching scope and cover more types of content, the SPB team
 developed the global search tool. This tool unifies search mechanisms
-provided by the mentioned collaborative softwares. Any user can
-find a public content in the context of social network, mailing list and
+provided by the different collaborative software used in the SPB protal. Any user can
+find a public content in the context of social networking, mailing list and
 software development.
 \subsection{Software development tools}
 The new SPB also provides
-tools to encourage developers to keep each source code and its
-developments within the platform. Any created software has, by default, a
-related git repository with wiki pages and issues tracking. These tools are
+tools to encourage developers to keep source code and its
+development activity within the platform. Any created software has, by default, a
+an associated git repository with wiki pages and issue tracking. These tools are
 supplied by the integration of Gitlab into the platform. 
 Developers can also evaluate the software source code to measure software
@@ -67,7 +67,7 @@ public, which allows greater transparency between the developer and the
 community that uses the software.  Thereby, the maintainers can decide if the
 given solution meets the source code quality requirements.
-Thus, the SPB becomes a platform to stimulate the openness of the source code;
+Thus, the SPB became a platform to stimulate the openness of the source code;
 dialogue between users and the development team; and also
 maintenance and evolution of the software, which will provide more
 transparency in government investments.
@@ -5,19 +5,19 @@ The SPB team was composed of a variety of professionals with different levels
 and skills, where most of them were undergraduate students with major in
 software engineering (from 4th semester or upper).  Since the students could
 not dedicate many hours per week to the project, they always had the
-flexibility to negotiate their work schedule during the semester in order not
-to cause any damage to their grades. Their daily work routine in the project
-included programming and devops tasks. 
+flexibility to negotiate their work schedule during the semester in
+order to not harm their classes and coursework. Their daily work routine
+in the project included programming and DevOps tasks.
 The development of SPB project required a vast experience and background that
-usually undergraduate students do not have yet. For this reason, some senior
-developers have joined to the project to help with hard issues and to transfer
-knowledge to the students. Their main task was to provide solutions for complex
-problems, in other words, they worked as a developer. As these professionals
-are very skillful and the project could not fund a full time work for them,
-some of them worked partially on the project. In addition, they lived in a
-different states spread around Brazil which led much of the communication to be
-made via Internet.
+usually undergraduate students do not have yet. For this reason, a few senior
+developers have joined to the project to help with the more difficult issues
+and to transfer knowledge to the students. Their main task was to provide
+solutions for complex problems, in other words, they worked as developers. As
+these professionals are very skillful and the project could not fund a full
+time work for them, some of them worked partially on the project. In addition,
+they lived in a different states spread around Brazil which led much of the
+communication to be online.
 In short, our work process was based on open and collaborative software
 development practices. The development process was defined based on the
@@ -26,108 +26,143 @@ high degree of automation resulting from DevOps practices. Thus, the work
 process was executed in a cadenced and continuous way.
 Finally, the last group of actors of this project was composed of employees
-formally working for the Brazilian government, in the Ministery of Planning,
-Development, and Management (MP is the Brazilian acronyms). All the project
+formally working for the Brazilian government, in the Ministry of Planning,
+Development, and Management (MP is the Brazilian acronym). All the project
 decisions, validations, and scope definitions were made by them. In this way we
-developed software product increments, releases,  aligned with business
-strategic objectives. As can be seen, the project had many kinds of profiles
-that had to be organized and synchronized.
+developed software product incrementally, with releases aligned to business
+strategic objectives. As one can see, the project had many different
+kinds of stakeholders that had to be organized and synchronized.
-\subsection{Teams organizations}
+\subsection{Team organization}
 Approximately 70\% of the development teams were composed of software
-engineering undergraduate students from UnB and they worked physically in the
-same laboratory in the opposite of the senior. Each student had their own
-scheduler based on their class, it made complicated to implement pair
-programming. Also, they had a different area of interests. To cope with those
-diversity, we had two basic rules which guided the project organization:
+engineering undergraduate students from UnB and they worked physically
+in the same laboratory. Each student had their own schedule based on
+their classes, what complicates the implementation of pair programming.
+The senior developers tried to synchronize their schedule with the
+students on their sub-team. To  cope with this environment, we had a few
+basic rules which guided the project organization:
 \begin{enumerate}
-  \item Classes have to be the high priority for undergraduate students;
-  \item Always work in pair (locally or remotely).
+  \item Classes have high priority for undergraduate students;
+  \item Work in pairs whenever possible (locally or remotely).
+  \item There must be one morning or afternoon per week when
+    \emph{everyone} should be together physically in the laboratory
+    (except of course the remote team members).
+  \item Every 3 to 4 months, the senior developers would fly in and work
+    alongside the students for a few days.
 \end{enumerate}
-With the aforementioned rules we divided all the project into four different
-teams: Colab, Noosfero, Design, and DevOps. Each team had one coach responsible
-for reducing the communication problem with the other teams and help the
-members to organize itself in the best way for everyone (always respecting the
-work time). The coach, was a normal student working in some of the teams with
-the extra duty of register the current tasks developed in the sprint and with
-the responsibility to talk with other teams. One important thing to notice is
-the mutability of the team and the coach, during the project many students
-changed between the teams to try different areas.
-
-One characteristic of the teams was the presence of (at least) one senior per
-team. This was essential, because hard decisions and complex problems were
-usually addressed to them, this relieved the coach duty to take a complicated
-technical decisions and encouraged students to be a coach. Lastly, the senior
-had to respect a rule number two and work with students, this was important to
-gave the undergraduate the opportunity to interact with a savvy professional in
-his area and keeping the knowledge flow in the project.
-
-Finally, we had to add two last elements of the team organization, that was
-essential for the project harmony: the meta-coach and professors. The former
-was a software engineer recently graduated and which wanted to keep working on
-the project, the latter were professors that orchestrated all the interactions
-between all members of the project. The meta-coach usually worked in one
-specific team and had the extra task of knowing the current status of all
-teams. Professors and meta-coaches worked together to reduce the communication
-problem between all the teams. Lastly, all the bureaucracy tasks was
-centralized in the professors.
-
-\subsection{Meetings}
-
-Brazilian government used to work with software development in a very
-traditional way, frequently they claim on documents and does not focus on what
-really matter (running software). This way of thinking caused to us a
-communication noise with MP, because they constantly tried to leverage on our
-work style. It was especially hard to convince them to accept the idea of open
-scope and agile development, but after months of labor and showing results they
-stopped resisting.
-
-We defined some level of meeting granularity to avoid to generate overheads to
-the developers. We had a strategical and validating meeting with MP (the
-former once in a month and the latter each 15th day), release plaining with the
-entire team (one per month), and finally a sprint planning (one each 15th day).
-Figure \ref{fig:meeting} is a diagram that represents our meeting organization.
+With the aforementioned rules we divided all the project into four
+different teams: Colab, Noosfero, Design, and DevOps. Each team had one
+coach responsible for reducing the communication problem with the other
+teams and help the members to organize themselves in the best way for
+everyone (always respecting their work time). The coach was one of the
+students working in some of the teams with the extra duty of registering
+the current tasks developed in the sprint and with the responsibility to
+talk with other teams. One important thing to notice is the mutability
+of the team and the coach, during the project many students changed
+between the teams to try different areas.
+
+One characteristic of the teams was the presence of (at least) one
+senior per team. This was essential, because hard decisions and complex
+problems were usually referred to them. This kept having to take
+complicated technical decisions from the coach tole, what encouraged
+students to be coaches. Lastly, the senior developers worked directly
+with the students, and this was important to give the undergraduate the
+opportunity to interact with a savvy professional in his area and to
+keep the knowledge flowing in the project.
+
+Finally, we had to add two last elements of the team organization, that
+was essential for the project harmony: the meta-coach and professors.
+The former was a software engineer recently graduated and which wanted
+to keep working on the project, the latter were professors that
+orchestrated all the interactions between all members of the project.
+The meta-coach usually worked in one specific team and had the extra
+task of knowing the current status of all teams. Professors and
+meta-coaches worked together to reduce the communication problem between
+all the teams. Lastly, all the paperwork tasks, such as reporting on the
+project progress to the Ministry, was handled by the professors.
+
+\subsection{Communication and management}
+
+Our team had many people working together, and most of the seniors worked in a
+different city remotely. Also, we tried to keep our work completely clear to
+the Brazilian government and citizens interested in following the project. To
+handle those cases, we used a set of tools to communication and other to manage
+the project.
+
+For communication between member in different places, we used: Google
+handouts with tmate tool, IRC, and mailing lists. When one student had to
+work in pair with a senior, normally, they used Google hangout for
+communication and they shared a terminal session with tmate which allow
+them to share the same terminal, with both typing and seeing the screen.
+For questions and fast discussion, we used IRC. For general
+notification, we used the mailing lists.
+
+For managing the project we used the SPB Portal itself; first to validate it by
+ourselves, and also because it had all the required tools. We basically created
+one wiki page per release in the SPB Gitlab instance with a mapping between
+strategical, tactical, and operational views. In a practical point of view, one
+milestone per user history (feature), and one or more issues for addressing
+each feature. With this approach we achieved two important goals: keeping all
+the management as close possible to to the source code, and tracking every
+feature developed during the project.
+
+\subsection{High-level project management and reporting}
+
+The Brazilian government used to work with software development in a
+very traditional way. They would frequently focus on documents and not
+on what was, in our opinion, what really matters: working software. This
+dissonance caused us a communication noise with MP, because they would
+often question our work style.  It was especially hard to convince them
+to accept the idea of open scope and agile development, but after months
+of labor and showing results they stopped resisting.
+
+We defined some level of meeting granularity to avoid generating too
+much overhead to the developers. We had a strategical and validating
+meeting with MP (the former once in a month and the latter each 15th
+day), release plaining with the entire team (one per month), and finally
+a sprint planning (one each 15th day).  Figure \ref{fig:meeting} is a
+diagram that represents our meeting organization.
 \begin{figure}[hbt]
   \centering
-    \includegraphics[width=.75\linewidth]{figures/meeting_flows.png}
+    \includegraphics[width=\linewidth]{figures/meeting_flows.png}
   \caption{Meetings cycles}
   \label{fig:meeting}
 \end{figure}
-In the strategical meeting we usually defined the priorities and new features
-with the Brazilian government (we always had to negotiate next steps with
-them). Normally the professors, the coach of each team, the meta-coach, and
-some employees of the MP join in this meeting. We usually discussed what the
-team already produced since our last meeting, and we establish the new features
-for the next release.  Notice that just part of the team join in this meeting
-to avoid generating unnecessary overhead to the developers, but all the
-students interested to participate was allowed to join (many students wanted
-this experience during the project).
+In the strategical meeting we usually defined the priorities and new
+features with the Brazilian government. Normally the professors, the
+coach of each team, the meta-coach, and some employees of the MP would
+participate in this meeting. We usually discussed what the team already
+produced since our last meeting, and established the new features for
+the next release.  Notice that just part of the team would join this
+meeting to avoid generating unnecessary overhead to the developers, but
+all the students interested to participate were allowed to join (many
+students wanted this experience during the project).
 After the strategical meeting with Brazilian government agents, we had a
 planning turn with all teams together. In this part, each team worked together
-to convert the MP wishes into small parts which was represented by the epics of
+to convert the MP wishes into smaller parts which were represented by the epics of
 the release. Each coach was responsible for conducting the planning, and after
-that register it on the project wiki (the wiki provided by Gitlab). With this
+that record it on the project wiki (the wiki provided by Gitlab). With this
 epic, each 14th day the team have generated their sprint scheduler (with small
-achievements mapped in issues).
+achievements mapped to issues).
-To keep the Brazilian government always updated, we invited them to work with
-us to validate the new features in progress. Normally we had a meeting each
-15th day. Basically, this was our work flow, we always kept everything
-extremely open to the MP (our way of work and open source projects) and to the
-team.
+To keep the Brazilian government always updated, we invited them to work
+with us to validate the new features in progress. Normally we had a
+meeting each 15th day. Basically, this was our work flow, we always kept
+everything extremely open to the MP (our way of working, and the one
+often used by open source projects) and to the team.
-To keep the track of all of those things we used the SPB, especially the
-Gitlab. Basically, we had:
+To keep the track of all of those things we used the SPB itself,
+especially Gitlab. Basically, we had:
 \begin{enumerate}
   \item Project repository: We have one organization with many repositories
-  \item Milestones: Each milestone is used to register a release
+  \item Milestones: Each milestone was used to register a release
   \item Wiki: Each release has one page on wiki with the compilation of
     strategical meeting
   \item Issues: Each sprint planning generated issues, which we associated to
@@ -135,32 +170,6 @@ Gitlab. Basically, we had:
     with them. Finally each developer assigned the issue to itself.
 \end{enumerate}
-Notice that this workflow gave to us and to the Brazilian government agents a
-full traceability from high view of the feature to the low view (code). This
-provided to them a way to  validate all worked done and proof the concept that
-work with open source project can give a proper view to them check.
-
-\subsection{Tools for communication and management}
-
-Our team had many people worked together, and most of the seniors worked in a
-different city remotely. Also, we tried to keep our work completely clear to
-the Brazilian government and citizens interested in follow the project. To
-handle those cases, we used a set of tools to communication and other to manage
-the project.
-
-For communication between member in different places, we used: google-talk with
-tmate, IRC, and mailing-list. When one student had to work in pair with a
-senior, normally, they used google-hangout for communication and they shared a
-session with tmate which allow them to share the same terminal. For questions
-and fast discussion, we used IRC. For general notification, we used the
-mailing-list.
-
-For managing the project we used the SPB Portal to validate it by ourselves and
-because it had all the required tools. We basically create one wiki page per
-release in Gitlab, one milestone per sprint, and one or more issues for address
-one user history. With this approach we achieve two important things: keep all
-the management close to the source code and tracked every feature developed by
-the project.
-
-%TODO: Ainda falta adicionar a parte da visita dos seniors e o turno sagrado
-
+Notice that this workflow gave us and the Brazilian government agents
+full traceability from a high level view of each feature to the lowest
+level (code).
-\section{Contributing with Free Software Communities}
+\section{Contributions to the upstream communities}
 \label{sec:contributions}
 %- projeto feito do jeito certo com relação ao software livre (contribuições upstream etc)
@@ -9,24 +9,30 @@
 %* Coper, empacotamentos (obs), omniauth
-During the execution of this project we made several contributions from
-different levels to the communities we interacted with. This occurred due to
-our development process aligned with those of the respective communities. We
-used to discuss with upstream the features and bug fixes that we was working
-on, this kind of discussion improve the developers' technical solutions and
-allowed upstream to accept our contribution more easily.
-
-In Colab we helped upstream to redesign the entirely architecture, enabling the
-development of plugins to integrate new tools. We also added a feature that
-allowed Colab to run asynchronous tasks, which was a major improvement for us
-since we were developing a complex system. A migration to the latest Django
-version was made (web framework used by Colab). Moreover, we worked on RevProxy
-(the greatest Colab dependency) to put it in a good shape, fixing many bugs.
+During the execution of this project we made several contributions to
+the upstream communities we interacted with. This occurred due to our
+development process aligned with those of the respective communities.
+During development, we would explicitly discuss  the features and bug
+fixes that we were working on with the applicable upstream communities.
+This contributed to improve the developers technical solutions with
+expertise outside of our team, and make it easier for those changes to
+be accepted in the original projects. Having changes accepted upstream
+in turn makes our life easier as it minimizes the delta between our
+codebase and upstream's allowing us to upgrade and benefit from
+development work from others.
+
+In Colab, we helped upstream redesign the entirely architecture,
+enabling the development of plugins to integrate new tools. We also
+added a feature that allowed Colab to run asynchronous tasks, which was
+a major improvement for us since we were developing a complex system. A
+migration to the latest Django version was made (web framework used by
+Colab). Moreover, we worked on RevProxy (the more important dependency
+of Colab) to put it in a good shape, fixing many bugs.
 Gitlab was the tool that we made the least number of modifications. We
-contributed with some improvements related with configuration files and we
-developed a new omniauth plugin, which enables the user authentication in
-Gitlab via REMOTE\_USER HTTP header. This omniauth plugin was needed because
+contributed with some improvements related with configuration files and
+we developed a new plugin that enables user authentication in Gitlab
+through the REMOTE\_USER HTTP header. This plugin was needed because
 Colab uses this mechanism to manage the authentication.
 Noosfero was the tool that contemplated several functional requirements,
@@ -35,15 +41,11 @@ migrate to the latest Rails version (web framework used by Noosfero), enable
 the federation implementation (federation with other social networks), decouple
 the interface and the back-end, and so forth.
-We also contributed with some DevOps tools as well during the project. Some
-member of our team took the maintenance of some python libraries that we used
-to support our scripts to upload our packages to OBS (Open Build Service).
-Since we were composed by many teams with large number of developers we had
-some problems related with the tracking of our per team/software releases, the
-DevOps team did not know when was the right time to package that software or
-not. Thus we developed a tool called copr-status to keep tracked the version
-packaged and the version finished by the developers, basically this is a web
-interface that helps you to visualize the status of that package/software.
+We also made some contributions on the DevOps front. Some members of
+them team became maintainers of some python libraries that were used by
+our scripts to upload packages to OBS (Open Build Service). We developed
+a tool called copr-status to keep track of the different stages of the
+lifecycle of each of the individual packages we were working on.
 %TODO: Mezuro
 \section{Lessons Learned}
 \label{sec:lessons}
-The multidisciplinary composition of the development teams, mainly software
-engineers and designers, is necessary for the development of good software
-products, which naturally aim to meet the users needs. In the context of the
-SPB project, there were also stakeholders from different areas who composed the
-team of technicians and managers of the MP, as well as the administrative teams
-of UnB. This interaction with different professionals brought a great learning
-opportunity for the students, who had their first professional experience, even
-during their graduation course. On the other hand, the different perceptions of
-stakeholders generated high complexity of communications and expectations
-management, burdening too much the professors who were responsible for project
-management.
-
-The use of the Colab tool was a requirement required by the MP. They argueed
-that this tool presented functionalities that would allow MP managers to
-stimulate the participation of SPB service providers with gamefication
-practices. As we said, in order for Colab to perform the expected behavior in
-SPB its architecture had to be completely redefined and this caused in
-practice, (i) the considerable increase in the architectural complexity of the
-SPB and (ii) it was the subsystem that consumed the most effort and budget
-during development.
-
-Due to the computational complexity related to the SPB deployment, associated
-with the Brazilian government needs for product support, we made an effort to
-provide complete automation of the entire deployment procedure, a result of
-DevOps activities.  In this way, we encapsulate all this complexity and enable
-the deployment of new SPB releases through the execution of few commands, as
-registered in the project documentation. Although we have provided a high
-degree of automation, training workshops for the MP technical team, and a
-meticulous description of the procedures in the documentation, we observed that
-the MP technical staff invariably depended on our support to perform these
-procedures.
-
-From the point of view of management and development processes, we had to
-develop management strategies that would accommodate different organizational
-cultures. As reported, the MP has a functional hierarchical organizational
-culture, strongly supported by process management, typical of the traditional
-development paradigm. The UnB teams use a process based on agile manifest
-values and FOSS and agile community practices. So we created a process of
-"translation" of work done to communicate with MP managers who manage their
-portfolio based on the PMBoK processes. On the one hand, in the intermediary
-and final project's phases we have matured this process, mainly due to the
-perception of the results by MP, on the other hand, in the initial phase we had
-a lot of intervention in our work style, which ended up focusing strategic
-discussions for operational discussions. Again there was an overload in
-professors, who were responsible for maintaining the strategic alignment of the
-MP with the day to day development of the UnB team.
-
-Another importance factor for the students and maturing of the software
-engineering practices used in the project was the composition of the teams with
-the participation of experienced professionals from the FOSS communities. These
-professionals together with the professors promoted a work environment where
-the students could develop their skills in a didactic and practical way without
-being transferred the pressures for them. In addition, these experienced
-professionals were responsible for the most relevant technical decisions
-related to the SPB software product.
- 
+\textbf{How to involve students real-world projects, interacting with
+real customers.}
+%
+Our team was mainly composed of software engineering undergraduate
+student, who had the opportunity to interact with senior developers and
+designers inside the team, as well as with the team of technicians and
+managers from the Brazilian Government, and the management team from
+UnB.
+%
+They interacted with professionals that had diverse expertises, and were
+able to participate in all levels of the software development process.
+This contribted to a great learning opportunity, and for a majority of
+them this was their first professional experience.
+
+\textbf{The participation of experienced professionals is crucial to
+success of the project.}
+One important factor for the students was the composition of the teams
+with the participation of experienced professionals.
+%
+On the technical side, the senior developers and designers would handle
+the more difficult technical decisions, creating a work environment
+where the students could develop their skills in a didactic way without
+pressure.
+%
+On the management side, the active participation of professors -- who
+are, in the end, the ones responsible for the project -- is crucial to
+make sure student participation is conducted in a healthy way, and is an
+instance of leading by example.
+
+\textbf{A balanced relationship between academia and industry.}
+The experience of the SPB project led UnB to develop a work style which
+proved to be appropriate for an educational environment that brings
+academia and industry together.
+%
+The highest priority from the university's point of view is the
+students.  Considering this, the activities of the project were never
+prioritized to the detriment of classes and other pedagogical
+activities. In summary, we had students working at different times, part
+time, remotely or locally, always respecting their individual
+conditions, but doing the work in a collective, collaborative and open
+way.
+%
+And even under a potentially adverse environment, the project delivered
+the desired solution with success.
+%
+At the end of the project, we noted that the skills developed by the
+students were at the state of art of the software engineering practice.
+After the project ended, we had team members successfully embracing
+opportunities in public, private, national and international
+organizations, in addition to those students that went into
+entrepreneurship and opened their own companies.
+
+\textbf{Managing different organizational cultures.}
+In the beginning of the project, the Brazilian Government stakeholders
+had certain expectations about the development of project that, let's
+say, didn't exactly match our work based on agile and FOSS practices.
+%
+We had to develop strategies that would support different these
+organizational cultures. As reported in Section \ref{sec:process}, the
+MP is organized in a functional hierarchical organizational structure,
+typically, a  traditional development paradigm. Therefore, we had to
+create a translation process between our team and the MP managers who
+managed the project on their side assuming a traditional, waterfall
+process.
+
+\textbf{Manage higher-level and lower-level goals separately.}
+During the initial phase of the project the MP team would often bring
+strategic discussions to technical/operational meetings, where we were
+supposed to discuss practical, technical decisions.
+%
+This produced a highly complex communication and management environment,
+overloading the professors because they were supposed to be responsible
+for maintaining the strategic alignment of the MP synchronized with
+implementation plans of the development team, specially in light of the
+aforementioned culture mismatch. Mixing both concerns in the same
+discussions caused confusion on both sides.
+%
+Towards the middle and end of the project we were able to keep those
+concerns separated, what eased the work of everyone involved.
+
+\textbf{Living with ill-advised political decisions.}
+At the initial phases of the project, by political and personal
+motivation, the main stakeholders from the Brazilian government imposed
+the use of Colab to guide the development of the new SPB platform.  Our
+team was totally against the idea because we already knew that Colab was
+a very experimental project and its adoption could dramatically increase
+the project complexity.  Even though we provided technical reasons to
+not utilize Colab, MP was adamant and we had to manage this problem. As
+explained in section \ref{sec:architecture} we did massive changes to
+it, and at the end of the project we completely rewrote Colab and made
+it stable. It is important to notice that the MP compelled us to accept
+a technical issue based only on political interests, without considering
+all the resources that would be spent due to that decision. At the end
+of the project, we verified that Colab indeed consumed a vast amount of
+the budget and increased the project complexity. In the end of the
+project and after our analysis on the decision made by the MP, we
+understand that MP's managers are capable of ignoring technical reasons
+in favor to a political decision.
+
+\textbf{Consider sustainability from the beginning.}
+In the process of deploying the SPB platform in the MP structure, we had
+to interact with the MP technicians. We did several workshops, training
+and a meticulous documentation describing all the required procedures to
+update the platform, however, we realized that the MP technicians
+constantly avoid the responsibility. After noticing the aforementioned
+situation, we organized a specific DevOps that completely automated all
+the deployment procedure. We simplified all the platform deployment to a
+few steps: (1) initial configurations (just ssh configuration) and (2)
+the execution of simple commands to completely update the platform. By
+the end of the project, we observed that the MP technicians invariably
+still depended on our support to update the platform even with all the
+automation provided by us. We were sadly left with a feeling of
+uncertainty about the future of the platform after the project ended. In
+hindsight, we realize that the MP dedicated systems analysts and
+managers to the project, but not operations technicians. The later
+should have been more involved with the process so they could at least
+comfortable with managing the platform infrastructure.
+
 % * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
 %% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
 %% afirmação, embora eu e Paulo consigamos perceber isso.
-The experience of the SPB project led UnB to develop a model of team
-composition and work style that proved to be appropriate to the cararteristics
-of an education environment, bringing the academy closer to the industry. The
-highest priority from the university's point of view is the students.
-Considering this, the activities of the project were never prioritized to the
-detriment of the classes and other didactic-pedagogical activities. In short we
-had students working at different times, part time, remotely or presential,
-always respecting their individual conditions, but doing the work in a
-collective, collaborative and open way. At the end of the project we realized
-that the skills developed by the students empowered them with the state in the
-practice of software engineering. The members of the teams got opportunities to
-work in public, private, national and international organizations, in addition
-to those students they preferred entrepreneurship, opening their own companies.
+%===========
+% Conclusion
+%===========
+
+\section{Final remarks}
+
+The portal is available at \url{softwarepublico.gov.br}. All
+documentation, including detailed architecture and operation manuals are
+also available\footnote{\url{https://softwarepublico.gov.br/doc/}
+(in Portuguese only at the moment)}).
+%
+All the integrated tools are FOSS and our contributions were published
+in open repositories, available on the SPB Portal itself. We also
+contributed these features back to the respective communities: that
+benefits those communities, as well as us since we can share future
+development and maintenance effort with other organizations that
+participate in their projects.
+
+More discussion ...
+
+...
+
+...
+
+%* utilização do projeto para formação de recursos humanos (alunos)
+
+%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
+
+%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
+
+%* 69 projetos marcados como SPB, de 81 no total na plataforma.
+
+%* 47\% é desenvolvido em PHP.
+
+% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
+
+% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
+
+% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
+
+%* Debater economia de recursos em orgão públicos
@@ -12,12 +12,12 @@
 \begin{document}
 \sloppy
-\title{	Development experience report on the new \\ Brazilian Public Software Portal}
+\title{Lições aprendidas ao longo do desenvolvimento do novo \\ Portal da Software Público Brasileiro}
-\author{Paulo Meirelles\inst{1,3}, Antonio Terceiro\inst{2}, Hilmer Neri\inst{1},\\
-	Melissa Wen\inst{2}, Rodrigo Siqueira\inst{3}, Lucas Kanashiro\inst{3}}
+\author{Paulo Meirelles\inst{1,3}, Antonio Terceiro\inst{2}, Melissa Wen\inst{2},
+       \\Rodrigo Siqueira\inst{3}, Hilmer Neri\inst{1}}
-\address{Laboratory of Production, Research and Innovation in Software (LAPPIS)\
+\address{Laboratório de Produção, Pesquisa e Inovação em Sofware(LAPPIS)\
 	Faculdade UnB Gama (FGA) -- Universidade de Brasília (UnB)\\
 	Área Especial de Indústria Projeção A, Setor Leste -- Gama - DF -- Brasil
   	\email{\{paulormm,hilmer\}@unb.br}
@@ -26,10 +26,10 @@
 	Rua Marechal Floriano, 28, Canela, Salvador - BA -- Brasil
 	\email{\{terceiro,melissa\}@colivre.coop.br}
 \nextinstitute
-	Free Libre Open Source Competence Center (CCSL)\
+	Centro de Competência em Software Livre (CCSL)\
 	Instituto de Matemática e Estatística (IME) -- Universidade de São Paulo (USP)\\
 	Rua do Matão, 1010, Cidade Universitária -- São Paulo - SP -- Brasil
-	\email{\{siqueira,lkd\}@ime.usp.br}
+	\email{\{paulormm,siqueira\}@ime.usp.br}
 }
 \maketitle
	@@ -3,7 +3,7 @@		@@ -3,7 +3,7 @@
3	The Brazilian Public Software (SPB) is a program by the Brazilian Federal	3	The Brazilian Public Software (SPB) is a program by the Brazilian Federal
4	Government to foster the sharing and collaboration on Free and Open Source	4	Government to foster the sharing and collaboration on Free and Open Source
5	Software (FOSS) solutions for the public administration. In the one hand,	5	Software (FOSS) solutions for the public administration. In the one hand,
6	-Brazilian Public Softwares have some differences from FOSS projects, in	6	+Brazilian Public Software have some differences from FOSS projects, in
7	particular, the software is considered a public good and the Federal government	7	particular, the software is considered a public good and the Federal government
8	assumes some responsibilities related to its use. In the other hand, the	8	assumes some responsibilities related to its use. In the other hand, the
9	software development principles are the same: the trend towards	9	software development principles are the same: the trend towards
	@@ -22,8 +22,27 @@ the platform architecture, features, and the user experience efforts carried		@@ -22,8 +22,27 @@ the platform architecture, features, and the user experience efforts carried
22	out, we also discuss our work process, based on agile and free software	22	out, we also discuss our work process, based on agile and free software
23	development practices, and the lessons learned in 30 months work on the SPB	23	development practices, and the lessons learned in 30 months work on the SPB
24	project.	24	project.
		25	+\end{abstract}
25		26
26	-\textbf{Keywords:} Brazilian Public Software, Free/Libre/Open Source Software,
27	- Software Evolution, Integrated Platform	27	+\begin{resumo}
		28	+O Software Público Brasileiro (SPB) é um programa do Governo Federal brasileiro
		29	+para promover o compartilhamento e a colaboração de projetos de Software Livre
		30	+para a administração pública. Por um lado, um software público brasileiro tem
		31	+algumas diferenças em relação a um software livre, em especial, o fato do
		32	+software público ser considerado com um bem público and o Governo Federal
		33	+assumir algumas responsabilidades relacionadas ao seu uso. Por outro lado,
		34	+teoricamente, os princípios de desenvolvimento de software são os mesmos:
		35	+tendência a descentralização das decisões sobre o projeto, compartilhamento de
		36	+informações e do desenvolvimento (código), e interação contínua com seus
		37	+usuários.
		38	+%
		39	+Neste contexto, projetamos uma plataforma baseada na evolução e integração de
		40	+ferramentas existentes que promovem a colaboração. Atualmente, o portal SPB
		41	+disponibiliza funcionalidaeds modernas para o desenvolvimento colaborativo de
		42	+software, ajudando a administração pública brasileira a compartilhar suas
		43	+solução.
		44	+%
28		45
29	-\end{abstract}	46	+
		47	+
		48	+\end{resumo}
1	\section{Introduction}	1	\section{Introduction}
2	\label{sec:intro}	2	\label{sec:intro}
3		3
4	-During the last few decades, the Brazilian Federal Government has
5	-improved its software adoption and development processes. In 2003, the
6	-recommendation to adopt Free/Open Source Software (FOSS) become a public
7	-policy. In 2007, the Brazilian Government released a portal called
8	-Brazilian Public Software (\textit{Software Público Brasileiro} -- SPB,
9	-in Portuguese), with the goal of sharing FOSS projects developed by, or
10	-for, the Brazilian Government.
11	-
12	-The Brazilian legal instrument on software contracting
13	-(\textit{Instrução Normativa} 04/2012) mandates that public management
14	-must consult the SPB Portal to adopt a software solution. The	4	+During the last few decades, the Brazilian Federal Government has been
		5	+trying to change its software adoption and development processes. For
		6	+instance, in 2003, the recommendation to adopt Free and Open Source
		7	+Software (FOSS) become a public policy. In 2007, the Brazilian
		8	+Government released a portal named Brazilian Public Software
		9	+(\textit{Software Público Brasileiro} -- SPB, in Portuguese), with the
		10	+goal of sharing FOSS projects developed by, or for, the Brazilian
		11	+Government. Additionally, the Brazilian legal instrument on software
		12	+contracting (known as IN 04/2012) mandates that public agents must give
		13	+priority to solutions available in the SPB Portal. In short, the
15	acquisition of a proprietary solution must be explicitly justified by	14	acquisition of a proprietary solution must be explicitly justified by
16	-demonstrating that there is no suitable option in the SPB Portal.
17	-
18	-Since 2009, however, the SPB Portal was having several technical issues.
19	-The original codebase was not being developed anymore, and there was a
20	-large amount of technical debt to overcome. The system was a modified
21	-version of an existing FOSS platform that was not being developed
22	-anymore, and the portal maintenance was becoming harder and harder.
23	-
24	-From January 2014 to June 2016, a new platform for the SPB Portal was
25	-designed and developed by the University of Brasília (UnB) and the
26	-University of São Paulo (USP) in a partnership with the Brazilian
27	-Ministry of Budget, Planning, and Management(MP). This new Portal was
28	-designed as an integrated platform for collaborative software
29	-development. It includes functionality for social networking, mailing
30	-lists, version control system, and source code quality monitoring. In
31	-this paper, we present an overview of this new generation of the SPB
32	-Portal.
33	-
34	-The project was developed by a team of 3 professors, 6 professionals, 2
35	-masters students, and approximately 40 undergraduate students (not all of
36	-them at the same time, though -- graduations and other events triggered
37	-changes in the team).	15	+demonstrating that there is no suitable alternative in the SPB Portal.
		16	+In 2013, the Brazilian Federal Court issued a ruling document
		17	+(\textit{Acórdão 2314/2013}) about an audit survey regarding the use of
		18	+agile methodologies in software development contracts with the public
		19	+administration.
		20	+
		21	+Despite of that, in practice, FOSS or agile methodologies, that is,
		22	+collaborative and empirical software development methods are not widely
		23	+practiced and understood by the Brazilian government agents. Thus, the
		24	+hierarchical and traditional processes from the government and the lack
		25	+of expertise in real-world software development of its agents produces a
		26	+situation of inneficient software development contracts and
		27	+unjustifiable expending of taxpayers' money.
		28	+
		29	+% TODO: ^ references
		30	+
		31	+Since 2009, the SPB Portal was having several technical issues. The
		32	+original codebase was not being developed anymore, and there was a large
		33	+amount of technical debt to overcome. The system was a modified version
		34	+of an existing FOSS platform called
		35	+OpenACS\footnote{\url{http://openacs.org}}, and the old SPB portal was
		36	+not being updated anymore against the official OpenACS releases. In this
		37	+scenario, the portal maintenance was becoming harder and harder.
		38	+
		39	+After some events and meetings to collect requirements from the federal
		40	+government and from the society, a new platform for the SPB Portal was
		41	+developed, among January 2014 and June 2016, by the University of
		42	+Brasília (UnB) and the University of São Paulo (USP) in a partnership
		43	+with the Brazilian Ministry of Budget, Planning, and Management (MP). It
		44	+was designed as an integrated platform for collaborative software
		45	+development., and includes functionality for social networking, mailing
		46	+lists, version control system, and source code quality monitoring. To
		47	+coordinate and develop this project during 30 months, UnB received from
		48	+the Brazilian Federal Government a total of 2,619,965.00 BRL (about
		49	+750,000.00 USD in June 2016).
38		50
39	\begin{figure*}[hbt]	51	\begin{figure*}[hbt]
40	\centering	52	\centering
41	- \includegraphics[width=.9\linewidth]{figures/home-SPB.png}	53	+ \includegraphics[width=.9\linewidth]{figures/home-SPB_2.png}
42	\caption{The new SPB Portal.}	54	\caption{The new SPB Portal.}
43	\label{fig:spb}	55	\label{fig:spb}
44	\end{figure*}	56	\end{figure*}
45		57
		58	+The project was developed by a team of 3 professors, 2 masters students, and
		59	+approximately 50 undergraduate students (not all of them at the same time,
		60	+though -- graduations and other events triggered changes in the team) together
		61	+with 2 professional designers and 6 senior developers from the FOSS
		62	+communities. The professors and all undergraduate student were from UnB, and
		63	+the master students were from USP. Regarding the designers and senior
		64	+developers, 7 of 8 they were living outside of Brasília: Curitiba/Brazil, São
		65	+Paulo/Brazil, Ribeirão Preto/Brazil, Salvador/Brazil, Punta Cana/Dominican
		66	+Republic, and Montreal/Canada. In other words, we had a team working in
		67	+distributed collaborative virtual environment.
		68	+
46	Figure \ref{fig:spb} shows the home page of this integrated platform.	69	Figure \ref{fig:spb} shows the home page of this integrated platform.
47	-The development tried to be as faithful as possible to FOSS development.
48	All development was done in the open, and the changes we needed in the	70	All development was done in the open, and the changes we needed in the
49	-tools were contributed back to their communities.	71	+FOSS tools were contributed back to their respective communities. Our
		72	+process was based on agile practices and FOSS communities interaction.
		73	+We defined development cycles and released 5 versions of the new SPB
		74	+Portal. The first release (beta) was in September 2014, only 9 months
		75	+from the beginning of the project. The old portal was shut down down in
		76	+September 2015. Finally, the last version illustrated in Figure 1 was
		77	+released in June 2016.
		78	+
		79	+In this paper, we present an overview of this new generation of the SPB
		80	+Portal. This experience report shares our methodology and process to
		81	+develop this project working with the Brazilian federal government to
		82	+comply with its requirements at the same time to be as faithful as
		83	+possible to FOSS development. Moreover, we discuss several lessons
		84	+learned to provide a distributed collaborative virtual environment
		85	+involving a large undergraduate student team and remote senior
		86	+developers. Lastly, we released an unprecedented platform for the
		87	+Brazilian government applying empirical software development methods.
		88	+This case can help other projects overcome similar software engineering
		89	+challenges in the future, as well as illustrate how universities can
		90	+improve the real-world experience of their students by means of this
		91	+kind of project.
50		92
1	\section{Architecture}	1	\section{Architecture}
2	\label{sec:architecture}	2	\label{sec:architecture}
3		3
4	-Based on the great list of functional requirements desired by Brazilian Federal
5	-Government we decided to select some FOSS systems that already contemplate some
6	-of them and improve these systems. And bringing the idea of community, it is
7	-undoable build a platform to be used by communities, which is a complex
8	-scenario, using just one tool.	4	+Based on the extensive list of functional requirements defined by the
		5	+Brazilian Federal Government, we selected some FOSS systems to form our
		6	+solution. We looked for system that together could provide the largest
		7	+subset possible of the requirements, and were fully aware that we would
		8	+need to improve those systems in order to provide the rest. We were also
		9	+convinced that it would be impossible to provide all of those
		10	+requirements with a single tool.
9		11
10	-At the point of view of the architecture, two main requirements was brought by
11	-the Brazilian Federal Government for the new platform:	12	+From the point of view of the architecture, two main requirements was
		13	+brought by the Brazilian Federal Government for the new platform:
12		14
13	\begin{enumerate}	15	\begin{enumerate}
14	\item \textit{Integrate existing FOSS systems}, with minimal differences from	16	\item \textit{Integrate existing FOSS systems}, with minimal differences from
	@@ -17,69 +19,81 @@ the Brazilian Federal Government for the new platform:		@@ -17,69 +19,81 @@ the Brazilian Federal Government for the new platform:
17	systems, as well as centralized authentication.	19	systems, as well as centralized authentication.
18	\end{enumerate}	20	\end{enumerate}
19		21
20	-Adopting existing FOSS systems by the government could bring the benefit of
21	-improvements done by the upstream communities, and the maintenance effort. On
22	-the other hand, integrating different tools with distinct intent, it is not an
23	-easy task and it was important to have a consistent user interface which
24	-justifies the last requirement.	22	+Adopting existing FOSS systems and minimizing locally-made changes had
		23	+the objective of being able to upgrade to newer versions of the original
		24	+software, benefiting from improvements and maintenance done by the
		25	+existing project communities. Providing a consistent user interface on
		26	+top of those different tools was needed to make the transition between
		27	+the different systems seamless from the point of view of users, which
		28	+would be confused by jumping through completely different interfaces
		29	+while interacting with the portal.
25		30
26	For the first requirement, we identified four main systems that required	31	For the first requirement, we identified four main systems that required
27	-specialized teams for work in the integration process. The teams learned how to
28	-develop for their assigned systems and contributed to the original
29	-communities, so that the version we used were not significantly different from
30	-the original.	32	+specialized teams for work in the integration process. The teams learned
		33	+how to develop for their assigned systems and contributed to the
		34	+original communities, so that the version we used were not significantly
		35	+different from the original.
31		36
32	-At the end of the project, SPB portal was composed of more than ten systems
33	-among them we can highlight: Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix,
34	-Munin, and so forth. The following sections explained a little bit of Colab,
35	-Noosfero, Mezuro, and Gitlab (the main tools which we contributed). Below, we
36	-described how we integrated those tools and conclude with the deployment.	37	+At the end of the project, the SPB portal was composed of more than ten
		38	+systems, such as Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix, and
		39	+Munin. The remainder of this section describes the most relevant of
		40	+them, as well as how they were integrated into the platform.
37		41
38	\subsection{Colab}	42	\subsection{Colab}
39		43
40	-Colab integrates web applications as its main goal, the user of this composed
41	-system should not notice the change between the integrated applications. Colab
42	-was designed to use one plugin for each system under its domain, this is
43	-guaranteed by four levels of integration:	44	+Colab\footnote{\url{https://github.com/colab}} is a systems integration
		45	+platform for web applications. One of its goals is allowing different
		46	+applications to be combined in such a way that an user does not notice
		47	+the change between applications. For that, Colab provides facilities
		48	+for:
44		49
45	\begin{itemize}	50	\begin{itemize}
46	- \item Authentication
47	- \item Visual
48	- \item Events
49	- \item Data and search engine	51	+ \item Centralized authentication
		52	+ \item Visual consistency
		53	+ \item Relaying of events between applications
		54	+ \item Integrated search engine
50	\end{itemize}	55	\end{itemize}
51		56
52	-The aforementioned integrations levels were possible, because Colab works as a
53	-reverse proxy, therefore all external requests pass through it.
54	-
55	-Single Sign-On (SSO) is used to login users throughout all integrated
56	-applications. REMOTE\_USER HTTP header is sent to the applications and we
57	-expect that they know how to handle it, since this is a common authentication
58	-mechanism. The integrated applications should be on a local network to avoid
59	-security issues. With this the user will be able to navigate through the
60	-platform applications and will not be asked for authentication credentials.
61	-
62	-As Colab\footnote{\url{https://github.com/colab}} is a reverse proxy, it makes
63	-some HTML transformations in the HTTP response of integrated applications to
64	-provide a unified interface. Allows one to define some default templates to be
65	-used by all applications and overwrite when needed in its plugin. This approach
66	-allowed us to reuse some HTML pages which facilitates maintenance.
67	-
68	-A publish-subscribe implementation was used to handle events in the platform.
69	-Thus, if some application want to trigger some action in case that other
70	-application do something is possible. A registration of the desired events and
71	-implementation of the handlers must be done in the plugin of each application.
72	-This bring us many options to innovate in these integrations.
73	-
74	-A integrated search engine is provided by Colab. Each plugin specify which data
75	-will be indexed and will became available for the users, just need an simple
76	-implementation of how should perform the collection of data.	57	+Colab implements this integration by working as a reverse proxy for the
		58	+integration applications, that is, all external requests pass through
		59	+Colab before reaching them.
		60	+
		61	+Centralized authentication is handled by letting users authenticate to
		62	+Colab, which then sends a REMOTE\_USER HTTP header to applications,
		63	+which are expected to be pre-configured to accept that as an indication
		64	+of the current user (REMOTE\_USER is a standard authentication mechanism
		65	+for web applications). This allows users to be automatically identified
		66	+to each of the applications without having to enter credentials for each
		67	+of them.
		68	+%
		69	+Of course, this requires that the integrated applications are not
		70	+accessible on the public internet, otherwise malicious users could send
		71	+their own crafted REMOTE\_USER headers and impersonate any user.
		72	+
		73	+For the visual consistency, Colab is able to make transformations to the
		74	+HTML produced by the integrated applications, ir order to provide a
		75	+unified interface. It allows one to define default templates to be used
		76	+by all applications, as well as providing extra ones in a plugin. This
		77	+approach allowed us to reuse common HTML pages, what facilitates
		78	+maintenance.
		79	+
		80	+Colab also provides a publish-subscribe event system. This allows one of
		81	+the applications, or Colab itself, to trigger an action in another
		82	+application. For example, when a user registers with Colab, all
		83	+applications need to be notified in order to create their own internal
		84	+representation of that user account.
		85	+
		86	+Colab also provides an integrated search engine, which can be configured
		87	+to specify exactly what data needs to be indexed for each of the
		88	+applications, and how to direct the user to that piece of information on
		89	+the specific application.
77		90
78	Initially, Colab had support for a small set of applications (Trac, GNU	91	Initially, Colab had support for a small set of applications (Trac, GNU
79	-Mailman, and Apache Lucene) and all of them was hard-coded. Our team evolved
80	-Colab so that it can now receive plugins to add support for new applications
81	-with minimal changes to its existing core. We developed plugins to be used in
82	-the SPB platform: Noosfero, GitLab, and Mezuro.	92	+Mailman, and Apache Lucene) and support for them was hard-coded. Our
		93	+team evolved Colab so that it can now receive plugins that add support
		94	+new applications without the need of changes to the Colab core. We also
		95	+developed plugins to be used in the SPB platform: Noosfero, GitLab, and
		96	+Mezuro.
83		97
84	\subsection{Noosfero}	98	\subsection{Noosfero}
85		99
	@@ -92,44 +106,40 @@ home pages and documentation, and contact forms.		@@ -92,44 +106,40 @@ home pages and documentation, and contact forms.
92		106
93	\subsection{Gitlab}	107	\subsection{Gitlab}
94		108
95	-GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository manager
96	-with wiki pages and issue tracking features. On the one hand, Github is the
97	-most famous and used Git repository manager. On the other hand, it is not a
98	-FOSS. Gitlab is a FOSS platform and focuses on delivering a holistic solution
99	-that will see developers from idea to production seamlessly and on a single
100	-platform. Thus, we have worked on it to integrated to Colab.
101	-
102	-Moreover, Gitlab has several features working properly than Github that is
103	-interesting in SPB context such as free for private projects, built-in
104	-continuous integration and continuous deployment with best practices, more
105	-control during downtime and upgrade, flexible permissions, Work-in-Progress
106	-protection, move issues between projects, Group-level milestones, Create new
107	-branches from issues, Issue board, Time tracking, as well new features and
108	-improvements every month on the 22nd.	109	+GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository
		110	+manager with wiki pages and issue tracking features. Gitlab is a FOSS
		111	+platform and focuses on delivering a holistic solution that will see
		112	+developers from idea to production seamlessly and on a single platform.
		113	+
		114	+Gitlab has several unique features, such as built-in continuous
		115	+integration and continuous deployment, flexible permissions, tracking of
		116	+Work-in-Progress work, moving issues between projects, group-level
		117	+milestones, creating new branches from issues, issues board, and time
		118	+tracking.
109		119
110	\subsection{Mezuro}	120	\subsection{Mezuro}
111		121
112	Mezuro\footnote{\url{http://mezuro.org/}} is a platform to collect source code	122	Mezuro\footnote{\url{http://mezuro.org/}} is a platform to collect source code
113	-metric to monitor the internal quality of softwares written in C, C++,
114	-Java, Python, Ruby, and PHP. GNU Mailman is used for mailing lists.	123	+metrics to monitor the internal quality of software written in C, C++,
		124	+Java, Python, Ruby, and PHP.
115		125
116	-In general, source code metric tools also do not present a friendly way to
117	-interpret its results and, even more, do not follow a standardization between
118	-them. Mezuro brings and presents these results to
119	-the end user, specially, by analyzing source code metric history during its
120	-life cycle.	126	+In general, source code metrics tools also do not present a friendly way
		127	+to interpret its results and, even more, do not follow a standardization
		128	+between them. Mezuro collects and presents these results to the end
		129	+user, specially, by analyzing source code metric history during its life
		130	+cycle.
121		131
122	-The Mezuro platform provides a single interface
123	-grouping available tools, allows selection and composition of metrics in a
124	-flexible manner, stores the metrics evolution history, presents
125	-results in a friendly way, as well as, allows users to customize the given
126	-interpretation accordingly to their own context.	132	+The Mezuro platform provides a single interface grouping available
		133	+tools, allows selection and composition of metrics in a flexible manner,
		134	+stores the metrics evolution history, presents results in a friendly
		135	+way, as well as, allows users to customize the given interpretation
		136	+accordingly to their own context.
127		137
128	\subsection{System unification}	138	\subsection{System unification}
129		139
130	\begin{figure}[hbt]	140	\begin{figure}[hbt]
131	\centering	141	\centering
132	- \includegraphics[width=.5\linewidth]{figures/arch.png}	142	+ \includegraphics[width=\linewidth]{figures/arch.png}
133	\caption{SPB architecture overview.}	143	\caption{SPB architecture overview.}
134	\label{fig:architecture}	144	\label{fig:architecture}
135	\end{figure}	145	\end{figure}
	@@ -143,42 +153,5 @@ functionality: instead of using the redundant restricted search		@@ -143,42 +153,5 @@ functionality: instead of using the redundant restricted search
143	functionality of each application, a search in the SPB portal might	153	functionality of each application, a search in the SPB portal might
144	return content from any of the applications, be it web pages, mailing	154	return content from any of the applications, be it web pages, mailing
145	list posts, or source code.	155	list posts, or source code.
146	-
147	-% Falar do devops
148	-\subsection{Deploy}
149	-
150	-The SPB platform was deployed in 7 virtual machines with different functions,
151	-as we can see in Figure \ref{fig:architecture2}.
152	-
153	-\begin{figure*}[hbt]
154	- \centering
155	- \includegraphics[width=.8\linewidth]{figures/arch2.png}
156	- \caption{Instanciation view of the SPB architecture.}
157	- \label{fig:architecture2}
158	-\end{figure*}
159	-
160	-The \textit{reverseproxy} handles the HTTP requests and redirects them to the
161	-\textit{integration}, the \textit{email} sends and receives e-mails on behalf
162	-of the platform and the \textit{monitor} keeps the entire environment tracked.
163	-These three \textit{VMs} mentioned - \textit{reverseproxy}, \textit{email} and
164	-\textit{monitor} - are accessible via Internet and the other ones are only
165	-available in the local network created between them.
166	-
167	-\textit{Integration} works as a second layer of proxy beneath
168	-\textit{reverseproxy}, any request to the platform will be handled by it. The
169	-Colab service provides interface, authentication and search engine integration
170	-among all the services. When a request is received to a specific service,
171	-Colab authenticates the user in the target tool, sends the request and makes a
172	-visual transformation in the HTML page which is the content of the response.
173	-Another user-oriented feature is the integrated search engine, when the user
174	-want to find something in the platform Colab will perform the search in the
175	-whole databases. Colab itself provides a web interface for GNU Mailman and we
176	-have two others integrated tools in \textit{integration}: Gitlab and Prezento.
177	-Gitlab provides web interface for Git repositories and issues tracker, and
178	-Prezento is a front-end for source code static analysis.
179	-
180	-The source code static analysis is performed by \textit{mezuro}. It runs some
181	-static analysis tools on source code stored in repository and provide this data
182	-to Prezento. A social network and CMS (Content Manager System) is provided by
183	-Noosfero in \textit{social}, and the databases of all tools with a cache
184	-service are in \textit{database}.	156	+%
		157	+The SPB platform was deployed in 7 virtual machines with different functions.
1	\section{Lessons Learned}	1	\section{Lessons Learned}
2	\label{sec:lessons}	2	\label{sec:lessons}
3		3
4	-The multidisciplinary composition of the development teams, mainly software
5	-engineers and designers, is necessary for the development of good software
6	-products, which naturally aim to meet the users needs. In the context of the
7	-SPB project, there were also stakeholders from different areas who composed the
8	-team of technicians and managers of the MP, as well as the administrative teams
9	-of UnB. This interaction with different professionals brought a great learning
10	-opportunity for the students, who had their first professional experience, even
11	-during their graduation course. On the other hand, the different perceptions of
12	-stakeholders generated high complexity of communications and expectations
13	-management, burdening too much the professors who were responsible for project
14	-management.
15	-
16	-The use of the Colab tool was a requirement required by the MP. They argueed
17	-that this tool presented functionalities that would allow MP managers to
18	-stimulate the participation of SPB service providers with gamefication
19	-practices. As we said, in order for Colab to perform the expected behavior in
20	-SPB its architecture had to be completely redefined and this caused in
21	-practice, (i) the considerable increase in the architectural complexity of the
22	-SPB and (ii) it was the subsystem that consumed the most effort and budget
23	-during development.
24	-
25	-Due to the computational complexity related to the SPB deployment, associated
26	-with the Brazilian government needs for product support, we made an effort to
27	-provide complete automation of the entire deployment procedure, a result of
28	-DevOps activities. In this way, we encapsulate all this complexity and enable
29	-the deployment of new SPB releases through the execution of few commands, as
30	-registered in the project documentation. Although we have provided a high
31	-degree of automation, training workshops for the MP technical team, and a
32	-meticulous description of the procedures in the documentation, we observed that
33	-the MP technical staff invariably depended on our support to perform these
34	-procedures.
35	-
36	-From the point of view of management and development processes, we had to
37	-develop management strategies that would accommodate different organizational
38	-cultures. As reported, the MP has a functional hierarchical organizational
39	-culture, strongly supported by process management, typical of the traditional
40	-development paradigm. The UnB teams use a process based on agile manifest
41	-values and FOSS and agile community practices. So we created a process of
42	-"translation" of work done to communicate with MP managers who manage their
43	-portfolio based on the PMBoK processes. On the one hand, in the intermediary
44	-and final project's phases we have matured this process, mainly due to the
45	-perception of the results by MP, on the other hand, in the initial phase we had
46	-a lot of intervention in our work style, which ended up focusing strategic
47	-discussions for operational discussions. Again there was an overload in
48	-professors, who were responsible for maintaining the strategic alignment of the
49	-MP with the day to day development of the UnB team.
50	-
51	-Another importance factor for the students and maturing of the software
52	-engineering practices used in the project was the composition of the teams with
53	-the participation of experienced professionals from the FOSS communities. These
54	-professionals together with the professors promoted a work environment where
55	-the students could develop their skills in a didactic and practical way without
56	-being transferred the pressures for them. In addition, these experienced
57	-professionals were responsible for the most relevant technical decisions
58	-related to the SPB software product.
59	-	4	+\textbf{How to involve students real-world projects, interacting with
		5	+real customers.}
		6	+%
		7	+Our team was mainly composed of software engineering undergraduate
		8	+student, who had the opportunity to interact with senior developers and
		9	+designers inside the team, as well as with the team of technicians and
		10	+managers from the Brazilian Government, and the management team from
		11	+UnB.
		12	+%
		13	+They interacted with professionals that had diverse expertises, and were
		14	+able to participate in all levels of the software development process.
		15	+This contribted to a great learning opportunity, and for a majority of
		16	+them this was their first professional experience.
		17	+
		18	+\textbf{The participation of experienced professionals is crucial to
		19	+success of the project.}
		20	+One important factor for the students was the composition of the teams
		21	+with the participation of experienced professionals.
		22	+%
		23	+On the technical side, the senior developers and designers would handle
		24	+the more difficult technical decisions, creating a work environment
		25	+where the students could develop their skills in a didactic way without
		26	+pressure.
		27	+%
		28	+On the management side, the active participation of professors -- who
		29	+are, in the end, the ones responsible for the project -- is crucial to
		30	+make sure student participation is conducted in a healthy way, and is an
		31	+instance of leading by example.
		32	+
		33	+\textbf{A balanced relationship between academia and industry.}
		34	+The experience of the SPB project led UnB to develop a work style which
		35	+proved to be appropriate for an educational environment that brings
		36	+academia and industry together.
		37	+%
		38	+The highest priority from the university's point of view is the
		39	+students. Considering this, the activities of the project were never
		40	+prioritized to the detriment of classes and other pedagogical
		41	+activities. In summary, we had students working at different times, part
		42	+time, remotely or locally, always respecting their individual
		43	+conditions, but doing the work in a collective, collaborative and open
		44	+way.
		45	+%
		46	+And even under a potentially adverse environment, the project delivered
		47	+the desired solution with success.
		48	+%
		49	+At the end of the project, we noted that the skills developed by the
		50	+students were at the state of art of the software engineering practice.
		51	+After the project ended, we had team members successfully embracing
		52	+opportunities in public, private, national and international
		53	+organizations, in addition to those students that went into
		54	+entrepreneurship and opened their own companies.
		55	+
		56	+\textbf{Managing different organizational cultures.}
		57	+In the beginning of the project, the Brazilian Government stakeholders
		58	+had certain expectations about the development of project that, let's
		59	+say, didn't exactly match our work based on agile and FOSS practices.
		60	+%
		61	+We had to develop strategies that would support different these
		62	+organizational cultures. As reported in Section \ref{sec:process}, the
		63	+MP is organized in a functional hierarchical organizational structure,
		64	+typically, a traditional development paradigm. Therefore, we had to
		65	+create a translation process between our team and the MP managers who
		66	+managed the project on their side assuming a traditional, waterfall
		67	+process.
		68	+
		69	+\textbf{Manage higher-level and lower-level goals separately.}
		70	+During the initial phase of the project the MP team would often bring
		71	+strategic discussions to technical/operational meetings, where we were
		72	+supposed to discuss practical, technical decisions.
		73	+%
		74	+This produced a highly complex communication and management environment,
		75	+overloading the professors because they were supposed to be responsible
		76	+for maintaining the strategic alignment of the MP synchronized with
		77	+implementation plans of the development team, specially in light of the
		78	+aforementioned culture mismatch. Mixing both concerns in the same
		79	+discussions caused confusion on both sides.
		80	+%
		81	+Towards the middle and end of the project we were able to keep those
		82	+concerns separated, what eased the work of everyone involved.
		83	+
		84	+\textbf{Living with ill-advised political decisions.}
		85	+At the initial phases of the project, by political and personal
		86	+motivation, the main stakeholders from the Brazilian government imposed
		87	+the use of Colab to guide the development of the new SPB platform. Our
		88	+team was totally against the idea because we already knew that Colab was
		89	+a very experimental project and its adoption could dramatically increase
		90	+the project complexity. Even though we provided technical reasons to
		91	+not utilize Colab, MP was adamant and we had to manage this problem. As
		92	+explained in section \ref{sec:architecture} we did massive changes to
		93	+it, and at the end of the project we completely rewrote Colab and made
		94	+it stable. It is important to notice that the MP compelled us to accept
		95	+a technical issue based only on political interests, without considering
		96	+all the resources that would be spent due to that decision. At the end
		97	+of the project, we verified that Colab indeed consumed a vast amount of
		98	+the budget and increased the project complexity. In the end of the
		99	+project and after our analysis on the decision made by the MP, we
		100	+understand that MP's managers are capable of ignoring technical reasons
		101	+in favor to a political decision.
		102	+
		103	+\textbf{Consider sustainability from the beginning.}
		104	+In the process of deploying the SPB platform in the MP structure, we had
		105	+to interact with the MP technicians. We did several workshops, training
		106	+and a meticulous documentation describing all the required procedures to
		107	+update the platform, however, we realized that the MP technicians
		108	+constantly avoid the responsibility. After noticing the aforementioned
		109	+situation, we organized a specific DevOps that completely automated all
		110	+the deployment procedure. We simplified all the platform deployment to a
		111	+few steps: (1) initial configurations (just ssh configuration) and (2)
		112	+the execution of simple commands to completely update the platform. By
		113	+the end of the project, we observed that the MP technicians invariably
		114	+still depended on our support to update the platform even with all the
		115	+automation provided by us. We were sadly left with a feeling of
		116	+uncertainty about the future of the platform after the project ended. In
		117	+hindsight, we realize that the MP dedicated systems analysts and
		118	+managers to the project, but not operations technicians. The later
		119	+should have been more involved with the process so they could at least
		120	+comfortable with managing the platform infrastructure.
		121	+
60	% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados	122	% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
61	%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa	123	%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
62	%% afirmação, embora eu e Paulo consigamos perceber isso.	124	%% afirmação, embora eu e Paulo consigamos perceber isso.
63		125
64	-The experience of the SPB project led UnB to develop a model of team
65	-composition and work style that proved to be appropriate to the cararteristics
66	-of an education environment, bringing the academy closer to the industry. The
67	-highest priority from the university's point of view is the students.
68	-Considering this, the activities of the project were never prioritized to the
69	-detriment of the classes and other didactic-pedagogical activities. In short we
70	-had students working at different times, part time, remotely or presential,
71	-always respecting their individual conditions, but doing the work in a
72	-collective, collaborative and open way. At the end of the project we realized
73	-that the skills developed by the students empowered them with the state in the
74	-practice of software engineering. The members of the teams got opportunities to
75	-work in public, private, national and international organizations, in addition
76	-to those students they preferred entrepreneurship, opening their own companies.	126	+%===========
		127	+% Conclusion
		128	+%===========
		129	+
		130	+\section{Final remarks}
		131	+
		132	+The portal is available at \url{softwarepublico.gov.br}. All
		133	+documentation, including detailed architecture and operation manuals are
		134	+also available\footnote{\url{https://softwarepublico.gov.br/doc/}
		135	+(in Portuguese only at the moment)}).
		136	+%
		137	+All the integrated tools are FOSS and our contributions were published
		138	+in open repositories, available on the SPB Portal itself. We also
		139	+contributed these features back to the respective communities: that
		140	+benefits those communities, as well as us since we can share future
		141	+development and maintenance effort with other organizations that
		142	+participate in their projects.
		143	+
		144	+More discussion ...
		145	+
		146	+...
		147	+
		148	+...
		149	+
		150	+%* utilização do projeto para formação de recursos humanos (alunos)
		151	+
		152	+%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
		153	+
		154	+%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
		155	+
		156	+%* 69 projetos marcados como SPB, de 81 no total na plataforma.
		157	+
		158	+%* 47\% é desenvolvido em PHP.
		159	+
		160	+% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
		161	+
		162	+% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
		163	+
		164	+% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
		165	+
		166	+%* Debater economia de recursos em orgão públicos
77		167