\section{Lessons Learned and Conclusion}
\label{sec:lessons}

%%% Acho que podemos organizar em algo assim, explicitando a lição%%%
%(Lesson 1 -- Students in real-world project interacting with real customers)

The multidisciplinary of our development team - mainly composed of software
engineers undergraduate students, senior developers, and designers - makes that
naturally fit the project requirements and consequently achieve a high quality.
To collaborate to that, there was a stakeholder team of technicians and
managers from the Brazilian Government, as well as the management team from the
UnB. In particular at the being of the project, the different perceptions of
the Brazilian Government stakeholders generated a high complex communication
and management environment. Once we were practicing an empirical development
approach based on FOSS and agile methods, our development team also interacted
directly with the stakeholders to mitigate several moments of management
overhead, supporting the UnB professors those were responsible for the project
management. The interaction with professionals, that have different expertises,
and the participation in all levels of the software development process
contributed with a great learning opportunity for the students, which majority
have their first professional experience.


At the initial phases of the project, by political and personal motivation, the
main stakeholders from the Brazilian government imposed the use of Colab to
guide the development of the new SPB  platform.  Our team were totally against
the idea because we already knew that Colab was a very experimental project and
its adoption could dramatically increase the project complexity.  Even though
we provided technical reasons to not utilize Colab, MP enforces it to us and we
had to manage this problem. As explained in section \ref{sec:architecture} we
did a massive change in it, and at the end of the project we completely rewrite
Colab and made it stable. It is important to notice that the MP compelled us to
accept a technical issue based only on political interests, without considering
all the money they would spend with their decision. At the end of the project,
we verified that Colab consumed a vast amount of the budget and indeed
increased the project complexity. In the end of the project and after our
analyses on the decisions made by the MP, we understand that MP's managers are
capable of ignoring technical reasons in favor to a political decision.

In the process of deploying the SPB platform in the MP structure, we had to
interact with the MP's technicians. We did several workshops, training and a
meticulous documentation describing all the required procedures to update the
platform, however, we realized that the MP's technicians constantly avoid the
responsibility. After notice the aforementioned situation, we organized a
specific team to start a DevOps activities and interacts with all the UnB
teams. After a year of work, we completely automated all the deployment
procedure. We simplified all the platform implantation to a few steps: (1)
initial configurations (just ssh configuration) and (2) the execution of simple
commands to completely update the platform. In the end of the project, we
observed that the MP technical invariably depends on our support to update the
platform even with all the automation provided by us.

From the point of view of management and development processes, we had to
develop strategies that would support different organizational cultures. As
reported in Section \ref{sec:process}, the MP is organized in a functional
hierarchical organizational structure, typically, a  traditional development
paradigm. The UnB team works based on agile manifest values and FOSS community
practices. Therefore, we had to create a translation process to communicate
with MP managers who manage their project based on the PMBoK ideas. On the one
hand, in the intermediary and final project's phases, we had matured this
process, mainly due to the perception of the results by the MP, on the other
hand, in the initial phase we had a lot of intervention in our work style,
which ended up focusing  strategic discussions for operational discussions. The
aforementioned situation created an overload to the professors because they
were responsible for maintaining the strategic alignment of the MP synchronized
with the development team.

Another important factor for the students was the composition of the teams with
the participation of senior professionals from the FOSS communities. These
professionals and professors promoted a work environment where the students
could develop their skills in a didactic way without transferring any pressure
to the students. Addition, these experienced professionals were responsible for
the most relevant technical decisions related to the SPB software product.
 
% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
%% afirmação, embora eu e Paulo consigamos perceber isso.

The experience of the SPB project led UnB to develop a work style which proved
to be appropriated to the characteristics of an educational environment and
brings the academy closer to the industry. The highest priority from the
university's point of view is the students, considering this, the activities of
the project were never prioritized to the detriment of the classes and other
pedagogical activities. In summary, we had students working at different times,
part time, remotely or locally, always respecting their individual conditions,
but doing the work in a collective, collaborative and open way. At the end of
the project, we realized that the skills developed by the students empowered
them with the state in the practice of software engineering. The members of the
teams got opportunities to work in public, private, national and international
organizations, in addition to those students they preferred entrepreneurship,
opening their own companies.

%===========
% Conclusion
%===========

The portal is available at \url{softwarepublico.gov.br}. All
documentation, including detailed architecture and operation manuals are
also available\footnote{\url{https://softwarepublico.gov.br/doc/}
(in Portuguese only at the moment)}).
%
All the integrated tools are FOSS and our contributions were published
in open repositories, available on the SPB Portal itself. We also
contributed these features back to the respective communities: that
benefits those communities, as well as us since we can share future
development and maintenance effort with other organizations that
participate in their projects.


%* utilização do projeto para formação de recursos humanos (alunos)

%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate

%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)

%* 69 projetos marcados como SPB, de 81 no total na plataforma.

%* 47\% é desenvolvido em PHP.

% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).

% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.

% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.

%* Debater economia de recursos em orgão públicos