\section{Conclusion}
\label{sec:conclusion}

In this work, we presented and discussed issues experienced during a
government-funded project, in partnership with the University of Brasilia and
the University of São Paulo, to evolve the Brazilian Public Software Portal.
Our contributions are twofold. First, we present the strategy used to develop
and to deliver an unprecedented platform to Brazilian government. Second, based
on the results of the SPB Portal project, we point out that it is possible to
mitigate conflicts experienced in the development environment and to conciliate
governmental and academy cultures. To summarize our main contributions, we
answered in this section the three open questions those guided this paper.

\textbf{Which strategy could be used to integrate several existing FLOSS tools
to promote a collaborative software development?}
%
The SPB portal integrates more than ten FLOSS tools and provides several
features, such as social network, mailing list, version control, content
management and source code quality monitoring. Concerned with the platform
sustainability and maintainability, the aforementioned FLOSS tools were
integrated with minimum differences from their official versions and the new
developed features were sent upstream to ensure an alignment between the portal
systems and their respective official versions. In the integration process, the
main softwares were identified, specific teams were formed to work with each
one of them and each team was composed of students with different levels of
skills and at least one senior professional.

\textbf{How to involve students in real-world projects interacting with real
customers?}
%
In terms of mitigating conflicts, we tried to show that, as long as the
university can provide a healthy and challenging environment to its students,
one may conciliate studies and professional training in universities.
%
The students interacted with professionals of diverse fields of expertise, and
they were able to participate in all levels of the software development
process.  This contributed to a great learning opportunity.
%
In our work process, based on open and collaborative software development
practices, students could negotiate their work schedule as well as count on IT
professionals to solve development issues.
%
Among the students, we have defined coaches for each team and a meta-coach
(coach of whole project). All coaches, together with professors, have
intermediated the communication between client (the Brazilian Government)
and the rest of the group. 
After the end of the project, some students successfully
embraced opportunities in public and private sectors, within national borders
and abroad. Some other students went further and started their own companies.

\textbf{How to introduce typical FLOSS collaborative and agile practices in the
governmental development process?}
%
With some adaptations, it is
feasible to conciliate agile methodologies and FLOSS practices to develop
software to governmental organizations with functional hierarchical structures
that use traditional development paradigm.
%
Aiming at reducing client questions about workconclusion, a DevOps team was
created to automate all deploy process and also  to work in continuous
delivery. The Government was brought to our work environment and interacted
with our management and communication tools. For the project success, we
focused on providing a friendly working environment as well as on showing to
governmental agents another way to interact with the FLOSS community and the
university.

\subsection{Lessons Learned}

From the answers of our initial open questions, we also can highlighted six
lessons learned to make easier to share our experience during the development
of the new SPB Portal.

\textbf{The participation of experienced professionals is crucial to
the success of the project.} 
One important factor for the students was the composition of the teams with the
participation of experienced professionals.
%
On the technical side, the senior developers and designers would handle the
more difficult technical decisions, creating a work environment where the
students could develop their skills in a didactic way without pressure.
%
On the management side, the active participation of professors -- who are, in
the end, the ones responsible for the project -- is crucial to make sure
students participation is conducted in a healthy way, and it is an instance of
leading by example.

\textbf{A balanced relationship between academia and industry.}
The experience of the SPB project led UnB to develop a work style which proved
to be appropriate for an educational environment that brings academia and
industry together.
%
The highest priority from the university's point of view is the students.
Considering this, the activities of the project were never prioritized to the
detriment of classes and other pedagogical activities. In summary, we had
students working at different times, part time, remotely or locally, always
respecting their individual conditions, but doing the work in a collective,
collaborative and open way.
%
And even under a potentially adverse environment, the project delivered the
desired solution with success.

\textbf{Managing different organizational cultures.}
%
In the beginning of the project, the Brazilian Government stakeholders had
certain expectations about the project development that, let's say, did not
exactly match our work style based on agile and FLOSS practices.
%
We had to develop strategies that would support these different organizational
cultures. Therefore, we have adapted the process between our team
and the Government managers who managed the project on their side, assuming a
traditional waterfall process.

\textbf{Managing higher-level and lower-level goals separately.}
%
During the initial phase of the project, the Brazilian Government team has
brought strategic discussions to technical/operational meetings that were
supposed to be about practical technical decisions.
%
This produced a highly complex communication and management environment,
overloading the professors because they were supposed for maintaining the
Government strategy synchronized with the implementation plans of the
development team.  This was hard, especially because the aforementioned
cultural mismatch. Mixing both concerns in the same discussions caused
confusion on both sides.
%
From the middle of the project we were able to keep those concerns separated,
what eased the work of everyone involved.

\textbf{Living with ill-advised political decisions.}
%
At the initial phases of the project, by political and personal motivation, the
main stakeholders from the Brazilian Government imposed the use of Colab to
guide the development of the new SPB platform. Our team was totally against the
idea because we already knew that Colab was a very experimental project and its
adoption could dramatically increase the project complexity. Even though, we
provided technical reasons to not utilize Colab, the Government was adamant and
we had to manage this problem. We did massive changes to Colab, and at the end
of the project we have completely rewritten it to make it stable. It is
important to notice that the Government compelled us to accept a technical
decision based only on political interests, without considering all the
resources that would be spent due to that decision. At the end of the project,
we verified that Colab indeed consumed a vast amount of the budget and
increased the project complexity. After our analysis on the decision made by
the Government, we understand that some Brazilian Government managers are
capable of ignoring technical reasons in favor of political decisions.

\textbf{Consider sustainability from the beginning.}
%
In the process of deploying the SPB platform in the Brazilian Government
infrastructure we had to interact with the Government technicians. We did
several workshops, training and a meticulous documentation describing all the
required procedures to update the platform, however, we realized that they
constantly avoid the responsibility. After noticing the aforementioned
situation, we organized a DevOps team that completely automated all the
deployment procedure. We simplified all the platform deployment to a few steps:
(i) initial configurations (just ssh configuration) and (ii) the execution of
simple commands to completely update the platform. By the end of the project,
we observed that the Government technicians invariably still depended on our
support to update the platform even with all the automation provided by us. We
were sadly left with a feeling of uncertainty about the future of the platform
after the project ended. In hindsight, we realize that the Brazilian Government
dedicated system analysts and managers to the project, but not operations
technicians. The later should have been more involved with the process so they
could at least be comfortable in managing the platform infrastructure.

\subsection{Final Remarks and Future Work}

Ultimately, the SPB portal is in
production\footnote{\url{https://softwarepublico.gov.br}} and its full
documentation, including detailed architecture and operation manuals, is also
available\footnote{\url{https://softwarepublico.gov.br/doc/}}.
%
All the integrated tools are FLOSS and our contributions were published in open
repositories, available on the SPB Portal itself. We also contributed these
features back to the respective communities, which benefits both those
communities and us, since we can share future development and maintenance
effort with other organizations that participate in these projects.

Future work should use data produced by the project to validate and evaluate
how the used FLOSS and Agile practices have impacted the students and the
governmental development process. For this, we will conduce a
\textit{postmortem} analysis using the project open data and a survey targeting
the involved stakeholders.

%===========
% Conclusion
%===========

% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
%% afirmação, embora eu e Paulo consigamos perceber isso.


%* utilização do projeto para formação de recursos humanos (alunos)

%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate

%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)

%* 69 projetos marcados como SPB, de 81 no total na plataforma.

%* 47\% é desenvolvido em PHP.

% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).

% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.

% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.

%* Debater economia de recursos em orgão públicos