\section{Lessons Learned}
\label{sec:lessons}

\textbf{How to involve students real-world projects, interacting with
real customers.}
%
Our team was mainly composed of software engineering undergraduate
student, who had the opportunity to interact with senior developers and
designers inside the team, as well as with the team of technicians and
managers from the Brazilian Government, and the management team from
UnB.
%
They interacted with professionals that had diverse expertises, and were
able to participate in all levels of the software development process.
This contributed to a great learning opportunity, and for a majority of
them this was their first professional experience.

\textbf{The participation of experienced professionals is crucial to
success of the project.}
One important factor for the students was the composition of the teams
with the participation of experienced professionals.
%
On the technical side, the senior developers and designers would handle
the more difficult technical decisions, creating a work environment
where the students could develop their skills in a didactic way without
pressure.
%
On the management side, the active participation of professors -- who
are, in the end, the ones responsible for the project -- is crucial to
make sure student participation is conducted in a healthy way, and is an
instance of leading by example.

\textbf{A balanced relationship between academia and industry.}
The experience of the SPB project led UnB to develop a work style which
proved to be appropriate for an educational environment that brings
academia and industry together.
%
The highest priority from the university's point of view is the
students.  Considering this, the activities of the project were never
prioritized to the detriment of classes and other pedagogical
activities. In summary, we had students working at different times, part
time, remotely or locally, always respecting their individual
conditions, but doing the work in a collective, collaborative and open
way.
%
And even under a potentially adverse environment, the project delivered
the desired solution with success.
%
At the end of the project, we noted that the skills developed by the
students were at the state of art of the software engineering practice.
After the project ended, we had team members successfully embracing
opportunities in public, private, national and international
organizations, in addition to those students that went into
entrepreneurship and opened their own companies.

\textbf{Managing different organizational cultures.}
In the beginning of the project, the Brazilian Government stakeholders
had certain expectations about the development of project that, let's
say, didn't exactly match our work based on agile and FOSS practices.
%
We had to develop strategies that would support different these
organizational cultures. As reported in Section \ref{sec:process}, the
MP is organized in a functional hierarchical organizational structure,
typically, a  traditional development paradigm. Therefore, we had to
create a translation process between our team and the MP managers who
managed the project on their side assuming a traditional, waterfall
process.

\textbf{Manage higher-level and lower-level goals separately.}
During the initial phase of the project the MP team would often bring
strategic discussions to technical/operational meetings, where we were
supposed to discuss practical, technical decisions.
%
This produced a highly complex communication and management environment,
overloading the professors because they were supposed to be responsible
for maintaining the strategic alignment of the MP synchronized with
implementation plans of the development team, specially in light of the
aforementioned culture mismatch. Mixing both concerns in the same
discussions caused confusion on both sides.
%
Towards the middle and end of the project we were able to keep those
concerns separated, what eased the work of everyone involved.

\textbf{Living with ill-advised political decisions.}
At the initial phases of the project, by political and personal
motivation, the main stakeholders from the Brazilian government imposed
the use of Colab to guide the development of the new SPB platform.  Our
team was totally against the idea because we already knew that Colab was
a very experimental project and its adoption could dramatically increase
the project complexity.  Even though we provided technical reasons to
not utilize Colab, MP was adamant and we had to manage this problem. As
explained in section \ref{sec:architecture} we did massive changes to
it, and at the end of the project we completely rewrote Colab and made
it stable. It is important to notice that the MP compelled us to accept
a technical issue based only on political interests, without considering
all the resources that would be spent due to that decision. At the end
of the project, we verified that Colab indeed consumed a vast amount of
the budget and increased the project complexity. In the end of the
project and after our analysis on the decision made by the MP, we
understand that MP's managers are capable of ignoring technical reasons
in favor to a political decision.

\textbf{Consider sustainability from the beginning.}
In the process of deploying the SPB platform in the MP structure, we had
to interact with the MP technicians. We did several workshops, training
and a meticulous documentation describing all the required procedures to
update the platform, however, we realized that the MP technicians
constantly avoid the responsibility. After noticing the aforementioned
situation, we organized a specific DevOps that completely automated all
the deployment procedure. We simplified all the platform deployment to a
few steps: (1) initial configurations (just ssh configuration) and (2)
the execution of simple commands to completely update the platform. By
the end of the project, we observed that the MP technicians invariably
still depended on our support to update the platform even with all the
automation provided by us. We were sadly left with a feeling of
uncertainty about the future of the platform after the project ended. In
hindsight, we realize that the MP dedicated systems analysts and
managers to the project, but not operations technicians. The later
should have been more involved with the process so they could at least
comfortable with managing the platform infrastructure.

% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
%% afirmação, embora eu e Paulo consigamos perceber isso.

%===========
% Conclusion
%===========

\textbf{Final remarks.}
The portal is available at \url{softwarepublico.gov.br}. All
documentation, including detailed architecture and operation manuals are
also available\footnote{\url{https://softwarepublico.gov.br/doc/}
(in Portuguese only at the moment)}).
%
All the integrated tools are FOSS and our contributions were published
in open repositories, available on the SPB Portal itself. We also
contributed these features back to the respective communities: that
benefits those communities, as well as us since we can share future
development and maintenance effort with other organizations that
participate in their projects.


%* utilização do projeto para formação de recursos humanos (alunos)

%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate

%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)

%* 69 projetos marcados como SPB, de 81 no total na plataforma.

%* 47\% é desenvolvido em PHP.

% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).

% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.

% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.

%* Debater economia de recursos em orgão públicos