Re-organizing the conclusion section

Paulo Meireles
1 parent 6456cfa8
Showing 5 changed files with 222 additions and 219 deletions Show diff stats
opensym2017/content/06-architecture.tex
opensym2017/content/11-conclusion.tex
opensym2017/content/11-lessons.tex
opensym2017/content/12-conclusion.tex
opensym2017/spb.tex
@@ -131,7 +131,7 @@ as we can see in Figure \ref{fig:architecture2}.
 \begin{figure*}[hbt]
   \centering
-    \includegraphics[width=.8\linewidth]{figures/arch3.png}
+    \includegraphics[width=\linewidth]{figures/arch3.png}
   \caption{Instanciation view of the SPB architecture.}
   \label{fig:architecture2}
 \end{figure*}
@@ -0,0 +1,220 @@
+\section{Conclusion}
+\label{sec:conclusion}
+
+In this work, we presented and discussed issues experienced during a government-funded 
+project, in partnership with the University of Brasilia and the University of
+São Paulo, to evolve the Brazilian Public Software Portal. 
+Its contributions are twofold. First, we present the strategy used to develop
+and to deliver an unprecedented platform to Brazilian government. Second,
+based on the results of the SPB Portal project, we point out that it is
+possible to mitigate conflicts experienced in the development environment
+and to conciliate governmental and academy cultures. To summarize our main contributions, we answered in this section the three open questions those guided this paper.
+
+
+\textbf{Q1 -- Which strategy could be used to integrate several existing
+FLOSS tools to promote the collaborative software development?}
+%
+The SPB portal integrates more than 10 FLOSS tools and provides several features,
+such as social network, mailing list, version control, content management and
+source code quality monitoring. Concerned with the platform sustainability and
+maintainability, the aforementioned 10 FLOSS tools were integrated with minimum
+differences from their official versions and the new developed features were
+sent upstream to ensure an alignment between the portal systems and their
+respective official versions. In the integration process, the main softwares
+were identified, specific teams were formed to work with each one of them
+and each team was composed of students with different levels of skills and at
+least one senior professional.
+
+\textbf{Q2 -- How to involve students in real-world projects interacting with
+real customers?}
+%
+In terms of mitigating conflicts, we tried to show that, as long as the
+university can provide a healthy and challenging environment to its students,
+one may conciliate studies and professional training in universities.
+%
+The students interacted with professionals of diverse fields of expertise, and they were
+able to participate in all levels of the software development process.
+This contributed to a great learning opportunity.
+%
+In our work process, based on open and collaborative software development
+practices, students could negotiate their work schedule as well as count on IT
+professionals to solve development issues.
+%
+Among the students, we have defined coaches for each team and a meta-coach
+(coach of whole project). All coaches, together with professors, have
+intermediated the communication between client (Ministry of Planning of Brazil)
+and the rest of the group. 
+After the end of the project, some students successfully
+embraced opportunities in public and private sectors, within national borders
+and abroad. Some other students went further and started their own companies.
+
+\textbf{Q3 -- How to introduce the FLOSS collaborative and agile
+practices to governmental development process?}
+With some adaptations, what we called the ``translation processes'', it is feasible
+to conciliate agile methodologies and FLOSS practices to develop software to 
+governmental organizations with functional hierarchical structures that use 
+traditional development paradigm.
+%
+Aiming at reducing client questions about workconclusion, a DevOps front was
+created to automate all deploy process and also  to work in continuous
+delivery. The government was brought to our work environment and interacted
+with our management and communication tools. For the project success, we
+focused on providing a friendly working environment as well as on showing to
+governmental agents another way to interact with the FLOSS community and the
+university.
+
+\subsection{Lessons Learned}
+\label{sec:lessons}
+
+From the answers of our initial open questions, we can also highlighted six lessons learned to make easier to share our experience during the development of the new SPB Portal.
+
+\textbf{L1 -- The participation of experienced professionals is crucial to
+the success of the project.} 
+One important factor for the students was the composition of the teams
+with the participation of experienced professionals.
+%
+On the technical side, the senior developers and designers would handle
+the more difficult technical decisions, creating a work environment
+where the students could develop their skills in a didactic way without
+pressure.
+%
+On the management side, the active participation of professors -- who
+are, in the end, the ones responsible for the project -- is crucial to
+make sure students participation is conducted in a healthy way, and it is an
+instance of leading by example.
+
+\textbf{L2 -- A balanced relationship between academia and industry.}
+The experience of the SPB project led UnB to develop a work style which
+proved to be appropriate for an educational environment that brings
+academia and industry together.
+%
+The highest priority from the university's point of view is the
+students.  Considering this, the activities of the project were never
+prioritized to the detriment of classes and other pedagogical
+activities. In summary, we had students working at different times, part
+time, remotely or locally, always respecting their individual
+conditions, but doing the work in a collective, collaborative and open
+way.
+%
+And even under a potentially adverse environment, the project delivered
+the desired solution with success.
+%
+At the end of the project, we noted that the skills developed by the
+students were at the software engineering state of the art.
+After the project ended, we had team members successfully embracing
+opportunities in public, private, national, and international
+organizations, in addition to those students that
+opened their own companies.
+
+\textbf{L3 -- Managing different organizational cultures.}
+In the beginning of the project, the Brazilian Government stakeholders
+had certain expectations about the project development that, let's
+say, didn't exactly match our work style based on agile and FLOSS practices.
+%
+We had to develop strategies that would support these different
+organizational cultures. The
+MP is organized in a functional hierarchical organizational structure,
+typically adopting a traditional development paradigm. Therefore, we had to
+create a translation process between our team and the MP managers who
+managed the project on their side assuming a traditional waterfall
+process.
+
+\textbf{L4 -- Managing higher-level and lower-level goals separately.}
+During the initial phase of the project, the MP team has brought
+strategic discussions to technical/operational meetings that
+were supposed to be about practical technical decisions.
+%
+This produced a highly complex communication and management environment,
+overloading the professors because they were supposed 
+for maintaining the MP strategy synchronized with the
+implementation plans of the development team. This was hard, especially because the
+aforementioned cultural mismatch. Mixing both concerns in the same
+discussions caused confusion on both sides.
+%
+From the middle of the project we were able to keep those
+concerns separated, what eased the work of everyone involved.
+
+\textbf{L5 -- Living with ill-advised political decisions.}
+At the initial phases of the project, by political and personal
+motivation, the main stakeholders from the Brazilian government imposed
+the use of Colab to guide the development of the new SPB platform. Our
+team was totally against the idea because we already knew that Colab was
+a very experimental project and its adoption could dramatically increase
+the project complexity. Even though we provided technical reasons to
+not utilize Colab, the MP was adamant and we had to manage this problem. We did massive changes to
+Colab, and at the end of the project we have completely rewritten it to make
+it stable. It is important to notice that the MP compelled us to accept	
+a technical decision based only on political interests, without considering
+all the resources that would be spent due to that decision. At the end
+of the project, we verified that Colab indeed consumed a vast amount of
+the budget and increased the project complexity. At the end of the
+project and after our analysis on the decision made by the MP, we
+understand that MP managers are capable of ignoring technical reasons
+in favor of political decisions.
+
+\textbf{L6 -- Consider sustainability from the beginning.}
+In the process of deploying the SPB platform in the MP infrastructure we had
+to interact with the MP technicians. We did several workshops, training
+and a meticulous documentation describing all the required procedures to
+update the platform, however, we realized that the MP technicians
+constantly avoid the responsibility. After noticing the aforementioned
+situation, we organized a DevOps team that completely automated all
+the deployment procedure. We simplified all the platform deployment to a
+few steps: (1) initial configurations (just ssh configuration) and (2)
+the execution of simple commands to completely update the platform. By
+the end of the project, we observed that the MP technicians invariably
+still depended on our support to update the platform even with all the
+automation provided by us. We were sadly left with a feeling of
+uncertainty about the future of the platform after the project ended. In
+hindsight, we realize that the MP dedicated system analysts and
+managers to the project, but not operations technicians. The later
+should have been more involved with the process so they could at least be
+comfortable in managing the platform infrastructure.
+
+
+\textbf{Final remarks and future work}
+
+The portal is available at \url{softwarepublico.gov.br}. All
+documentation, including detailed architecture and operation manuals are
+also available\footnote{\url{https://softwarepublico.gov.br/doc/}
+(in Portuguese only at the moment)}.
+%
+All the integrated tools are FLOSS and our contributions were published
+in open repositories, available on the SPB Portal itself. We also
+contributed these features back to the respective communities, which
+benefits both those communities and us, since we can share future
+development and maintenance effort with other organizations that
+participate in these projects.
+
+Future work should use data produced by the project to validate and evaluate
+how the used FLOSS and Agile practices have impacted the students and the
+governmental development process. For this, we would conduce a \textit{postmortem}
+analysis using the project open data and a survey targeting the involved actors.
+
+%===========
+% Conclusion
+%===========
+
+% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
+%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
+%% afirmação, embora eu e Paulo consigamos perceber isso.
+
+
+%* utilização do projeto para formação de recursos humanos (alunos)
+
+%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
+
+%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
+
+%* 69 projetos marcados como SPB, de 81 no total na plataforma.
+
+%* 47\% é desenvolvido em PHP.
+
+% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
+
+% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
+
+% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
+
+%* Debater economia de recursos em orgão públicos
+
@@ -1,121 +0,0 @@
-\section{Lessons Learned}
-\label{sec:lessons}
-
-\textbf{How to involve students in real-world projects, interacting with
-real customers.}
-%
-Our team was mainly composed of software engineering undergraduate
-students, who had the opportunity to interact with senior developers and
-designers on the team, as well as with the team of technicians and
-managers from the Brazilian Government, and the management team from
-UnB.
-%
-The students interacted with professionals of diverse fields of expertise, and they were
-able to participate in all levels of the software development process.
-This contributed to a great learning opportunity. Moreover, for the majority of
-the students, this was a first professional experience.
-
-\textbf{The participation of experienced professionals is crucial to
-the success of the project.}
-One important factor for the students was the composition of the teams
-with the participation of experienced professionals.
-%
-On the technical side, the senior developers and designers would handle
-the more difficult technical decisions, creating a work environment
-where the students could develop their skills in a didactic way without
-pressure.
-%
-On the management side, the active participation of professors -- who
-are, in the end, the ones responsible for the project -- is crucial to
-make sure students participation is conducted in a healthy way, and it is an
-instance of leading by example.
-
-\textbf{A balanced relationship between academia and industry.}
-The experience of the SPB project led UnB to develop a work style which
-proved to be appropriate for an educational environment that brings
-academia and industry together.
-%
-The highest priority from the university's point of view is the
-students.  Considering this, the activities of the project were never
-prioritized to the detriment of classes and other pedagogical
-activities. In summary, we had students working at different times, part
-time, remotely or locally, always respecting their individual
-conditions, but doing the work in a collective, collaborative and open
-way.
-%
-And even under a potentially adverse environment, the project delivered
-the desired solution with success.
-%
-At the end of the project, we noted that the skills developed by the
-students were at the software engineering state of the art.
-After the project ended, we had team members successfully embracing
-opportunities in public, private, national, and international
-organizations, in addition to those students that
-opened their own companies.
-
-\textbf{Managing different organizational cultures.}
-In the beginning of the project, the Brazilian Government stakeholders
-had certain expectations about the project development that, let's
-say, didn't exactly match our work style based on agile and FLOSS practices.
-%
-We had to develop strategies that would support these different
-organizational cultures. As reported in Section \ref{sec:process}, the
-MP is organized in a functional hierarchical organizational structure,
-typically adopting a traditional development paradigm. Therefore, we had to
-create a translation process between our team and the MP managers who
-managed the project on their side assuming a traditional waterfall
-process.
-
-\textbf{Managing higher-level and lower-level goals separately.}
-During the initial phase of the project, the MP team has brought
-strategic discussions to technical/operational meetings that
-were supposed to be about practical technical decisions.
-%
-This produced a highly complex communication and management environment,
-overloading the professors because they were supposed 
-for maintaining the MP strategy synchronized with the
-implementation plans of the development team. This was hard, especially because the
-aforementioned cultural mismatch. Mixing both concerns in the same
-discussions caused confusion on both sides.
-%
-From the middle of the project we were able to keep those
-concerns separated, what eased the work of everyone involved.
-
-\textbf{Living with ill-advised political decisions.}
-At the initial phases of the project, by political and personal
-motivation, the main stakeholders from the Brazilian government imposed
-the use of Colab to guide the development of the new SPB platform. Our
-team was totally against the idea because we already knew that Colab was
-a very experimental project and its adoption could dramatically increase
-the project complexity. Even though we provided technical reasons to
-not utilize Colab, the MP was adamant and we had to manage this problem. As
-explained in section \ref{sec:architecture}, we did massive changes to
-Colab, and at the end of the project we have completely rewritten it to make
-it stable. It is important to notice that the MP compelled us to accept
-a technical decision based only on political interests, without considering
-all the resources that would be spent due to that decision. At the end
-of the project, we verified that Colab indeed consumed a vast amount of
-the budget and increased the project complexity. At the end of the
-project and after our analysis on the decision made by the MP, we
-understand that MP managers are capable of ignoring technical reasons
-in favor of political decisions.
-
-\textbf{Consider sustainability from the beginning.}
-In the process of deploying the SPB platform in the MP infrastructure we had
-to interact with the MP technicians. We did several workshops, training
-and a meticulous documentation describing all the required procedures to
-update the platform, however, we realized that the MP technicians
-constantly avoid the responsibility. After noticing the aforementioned
-situation, we organized a DevOps team that completely automated all
-the deployment procedure. We simplified all the platform deployment to a
-few steps: (1) initial configurations (just ssh configuration) and (2)
-the execution of simple commands to completely update the platform. By
-the end of the project, we observed that the MP technicians invariably
-still depended on our support to update the platform even with all the
-automation provided by us. We were sadly left with a feeling of
-uncertainty about the future of the platform after the project ended. In
-hindsight, we realize that the MP dedicated system analysts and
-managers to the project, but not operations technicians. The later
-should have been more involved with the process so they could at least be
-comfortable in managing the platform infrastructure.
-
@@ -1,95 +0,0 @@
-\section{Conclusion}
-\label{sec:conclusion}
-
-In this paper we presented and discussed issues experienced during a government-funded 
-project, in partnership with the University of Brasilia and the University of
-São Paulo, to evolve the Brazilian Public Software Portal. 
-Its contributions are twofold. First, we present the strategy used to develop
-and to deliver an unprecedented platform to Brazilian government. Second,
-based on the results of the SPB Portal project, we point out that it is
-possible to mitigate conflicts experienced in the development environment
-and to conciliate governmental and academy cultures. 
-
-The SPB portal integrates more than 10 FLOSS tools and provides several features,
-such as social network, mailing list, version control, content management and
-source code quality monitoring. Concerned with the platform susteinability and
-maintainabilty, the aforementioned 10 FLOSS tools were integrated with minimum
-differences from their official versions and the new developed features were
-sent upstream to ensure an alignment between the portal systems and their
-respective official versions. In the integration process, the main softwares
-were identified, specific teams were formed to work with each one of them
-and each team was composed of students with different levels of skills and at
-least one senior professional.
-
-In terms of mitigating conflicts, we tried to show that, as long as the
-institution can provide a healthy and challenging environment to its students,
-one may conciliate studies and professional training in universities.
-In our work process, based on open and collaborative software development
-practices, students could negotiate their work schedule as well as count on IT
-professionals to solve development issues.
-Among the students, we have defined coachs for each team and a meta-coach
-(coach of whole project). All coaches, together with professors, have
-intermediated the comunication between client (Ministry of Planning of Brasil)
-and the rest of the group. 
-After the end of the project, some students successfully
-embraced opportunities in public and private sectors, within national borders
-and abroad. Some other students went further and started their own companies.
-
-We also demonstrate that, with some adaptations/"translation processes", it is feasible
-to conciliate agile methodologies and FLOSS practices to develop software to 
-governmental organizations with functional hierarchical structures that use 
-traditional development paradigm.
-Aiming at reducing client questions about workconclusion, a DevOps front was
-created to automate all deploy process and also  to work in continuous
-delivery. The government was brought to our work environment and interacted
-with our management and comunication tools. For the project success, we
-focused on providing a friendly working environment as well as on showing to
-governmental agents another way to interact with the FLOSS community and the
-university. 
-
-Future work should use data produced by the project to validate and evaluate
-how the used FLOSS and Agile practices have impacted the students and the
-governmental development process. For this, we would conduce a \textit{postmortem}
-analysis using the project open data and a survey targeting the involved actors.
-
-\textbf{Final remarks}
-
-The portal is available at \url{softwarepublico.gov.br}. All
-documentation, including detailed architecture and operation manuals are
-also available\footnote{\url{https://softwarepublico.gov.br/doc/}
-(in Portuguese only at the moment)}.
-%
-All the integrated tools are FLOSS and our contributions were published
-in open repositories, available on the SPB Portal itself. We also
-contributed these features back to the respective communities, which
-benefits both those communities and us, since we can share future
-development and maintenance effort with other organizations that
-participate in these projects. 
-
-%===========
-% Conclusion
-%===========
-
-% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
-%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
-%% afirmação, embora eu e Paulo consigamos perceber isso.
-
-
-%* utilização do projeto para formação de recursos humanos (alunos)
-
-%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
-
-%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
-
-%* 69 projetos marcados como SPB, de 81 no total na plataforma.
-
-%* 47\% é desenvolvido em PHP.
-
-% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
-
-% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
-
-% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
-
-%* Debater economia de recursos em orgão públicos
-
@@ -187,8 +187,7 @@
 \input{content/08-ux}
 \input{content/09-process}
 \input{content/10-contributions}
-\input{content/11-lessons}
-\input{content/12-conclusion}
+\input{content/11-conclusion}
 %------------------------------------------------------------------------------
 \bibliographystyle{SIGCHI-Reference-Format}
	@@ -131,7 +131,7 @@ as we can see in Figure \ref{fig:architecture2}.		@@ -131,7 +131,7 @@ as we can see in Figure \ref{fig:architecture2}.
131		131
132	\begin{figure*}[hbt]	132	\begin{figure*}[hbt]
133	\centering	133	\centering
134	- \includegraphics[width=.8\linewidth]{figures/arch3.png}	134	+ \includegraphics[width=\linewidth]{figures/arch3.png}
135	\caption{Instanciation view of the SPB architecture.}	135	\caption{Instanciation view of the SPB architecture.}
136	\label{fig:architecture2}	136	\label{fig:architecture2}
137	\end{figure*}	137	\end{figure*}
@@ -0,0 +1,220 @@		@@ -0,0 +1,220 @@
	1	+\section{Conclusion}
	2	+\label{sec:conclusion}
	3	+
	4	+In this work, we presented and discussed issues experienced during a government-funded
	5	+project, in partnership with the University of Brasilia and the University of
	6	+São Paulo, to evolve the Brazilian Public Software Portal.
	7	+Its contributions are twofold. First, we present the strategy used to develop
	8	+and to deliver an unprecedented platform to Brazilian government. Second,
	9	+based on the results of the SPB Portal project, we point out that it is
	10	+possible to mitigate conflicts experienced in the development environment
	11	+and to conciliate governmental and academy cultures. To summarize our main contributions, we answered in this section the three open questions those guided this paper.
	12	+
	13	+
	14	+\textbf{Q1 -- Which strategy could be used to integrate several existing
	15	+FLOSS tools to promote the collaborative software development?}
	16	+%
	17	+The SPB portal integrates more than 10 FLOSS tools and provides several features,
	18	+such as social network, mailing list, version control, content management and
	19	+source code quality monitoring. Concerned with the platform sustainability and
	20	+maintainability, the aforementioned 10 FLOSS tools were integrated with minimum
	21	+differences from their official versions and the new developed features were
	22	+sent upstream to ensure an alignment between the portal systems and their
	23	+respective official versions. In the integration process, the main softwares
	24	+were identified, specific teams were formed to work with each one of them
	25	+and each team was composed of students with different levels of skills and at
	26	+least one senior professional.
	27	+
	28	+\textbf{Q2 -- How to involve students in real-world projects interacting with
	29	+real customers?}
	30	+%
	31	+In terms of mitigating conflicts, we tried to show that, as long as the
	32	+university can provide a healthy and challenging environment to its students,
	33	+one may conciliate studies and professional training in universities.
	34	+%
	35	+The students interacted with professionals of diverse fields of expertise, and they were
	36	+able to participate in all levels of the software development process.
	37	+This contributed to a great learning opportunity.
	38	+%
	39	+In our work process, based on open and collaborative software development
	40	+practices, students could negotiate their work schedule as well as count on IT
	41	+professionals to solve development issues.
	42	+%
	43	+Among the students, we have defined coaches for each team and a meta-coach
	44	+(coach of whole project). All coaches, together with professors, have
	45	+intermediated the communication between client (Ministry of Planning of Brazil)
	46	+and the rest of the group.
	47	+After the end of the project, some students successfully
	48	+embraced opportunities in public and private sectors, within national borders
	49	+and abroad. Some other students went further and started their own companies.
	50	+
	51	+\textbf{Q3 -- How to introduce the FLOSS collaborative and agile
	52	+practices to governmental development process?}
	53	+With some adaptations, what we called the ``translation processes'', it is feasible
	54	+to conciliate agile methodologies and FLOSS practices to develop software to
	55	+governmental organizations with functional hierarchical structures that use
	56	+traditional development paradigm.
	57	+%
	58	+Aiming at reducing client questions about workconclusion, a DevOps front was
	59	+created to automate all deploy process and also to work in continuous
	60	+delivery. The government was brought to our work environment and interacted
	61	+with our management and communication tools. For the project success, we
	62	+focused on providing a friendly working environment as well as on showing to
	63	+governmental agents another way to interact with the FLOSS community and the
	64	+university.
	65	+
	66	+\subsection{Lessons Learned}
	67	+\label{sec:lessons}
	68	+
	69	+From the answers of our initial open questions, we can also highlighted six lessons learned to make easier to share our experience during the development of the new SPB Portal.
	70	+
	71	+\textbf{L1 -- The participation of experienced professionals is crucial to
	72	+the success of the project.}
	73	+One important factor for the students was the composition of the teams
	74	+with the participation of experienced professionals.
	75	+%
	76	+On the technical side, the senior developers and designers would handle
	77	+the more difficult technical decisions, creating a work environment
	78	+where the students could develop their skills in a didactic way without
	79	+pressure.
	80	+%
	81	+On the management side, the active participation of professors -- who
	82	+are, in the end, the ones responsible for the project -- is crucial to
	83	+make sure students participation is conducted in a healthy way, and it is an
	84	+instance of leading by example.
	85	+
	86	+\textbf{L2 -- A balanced relationship between academia and industry.}
	87	+The experience of the SPB project led UnB to develop a work style which
	88	+proved to be appropriate for an educational environment that brings
	89	+academia and industry together.
	90	+%
	91	+The highest priority from the university's point of view is the
	92	+students. Considering this, the activities of the project were never
	93	+prioritized to the detriment of classes and other pedagogical
	94	+activities. In summary, we had students working at different times, part
	95	+time, remotely or locally, always respecting their individual
	96	+conditions, but doing the work in a collective, collaborative and open
	97	+way.
	98	+%
	99	+And even under a potentially adverse environment, the project delivered
	100	+the desired solution with success.
	101	+%
	102	+At the end of the project, we noted that the skills developed by the
	103	+students were at the software engineering state of the art.
	104	+After the project ended, we had team members successfully embracing
	105	+opportunities in public, private, national, and international
	106	+organizations, in addition to those students that
	107	+opened their own companies.
	108	+
	109	+\textbf{L3 -- Managing different organizational cultures.}
	110	+In the beginning of the project, the Brazilian Government stakeholders
	111	+had certain expectations about the project development that, let's
	112	+say, didn't exactly match our work style based on agile and FLOSS practices.
	113	+%
	114	+We had to develop strategies that would support these different
	115	+organizational cultures. The
	116	+MP is organized in a functional hierarchical organizational structure,
	117	+typically adopting a traditional development paradigm. Therefore, we had to
	118	+create a translation process between our team and the MP managers who
	119	+managed the project on their side assuming a traditional waterfall
	120	+process.
	121	+
	122	+\textbf{L4 -- Managing higher-level and lower-level goals separately.}
	123	+During the initial phase of the project, the MP team has brought
	124	+strategic discussions to technical/operational meetings that
	125	+were supposed to be about practical technical decisions.
	126	+%
	127	+This produced a highly complex communication and management environment,
	128	+overloading the professors because they were supposed
	129	+for maintaining the MP strategy synchronized with the
	130	+implementation plans of the development team. This was hard, especially because the
	131	+aforementioned cultural mismatch. Mixing both concerns in the same
	132	+discussions caused confusion on both sides.
	133	+%
	134	+From the middle of the project we were able to keep those
	135	+concerns separated, what eased the work of everyone involved.
	136	+
	137	+\textbf{L5 -- Living with ill-advised political decisions.}
	138	+At the initial phases of the project, by political and personal
	139	+motivation, the main stakeholders from the Brazilian government imposed
	140	+the use of Colab to guide the development of the new SPB platform. Our
	141	+team was totally against the idea because we already knew that Colab was
	142	+a very experimental project and its adoption could dramatically increase
	143	+the project complexity. Even though we provided technical reasons to
	144	+not utilize Colab, the MP was adamant and we had to manage this problem. We did massive changes to
	145	+Colab, and at the end of the project we have completely rewritten it to make
	146	+it stable. It is important to notice that the MP compelled us to accept
	147	+a technical decision based only on political interests, without considering
	148	+all the resources that would be spent due to that decision. At the end
	149	+of the project, we verified that Colab indeed consumed a vast amount of
	150	+the budget and increased the project complexity. At the end of the
	151	+project and after our analysis on the decision made by the MP, we
	152	+understand that MP managers are capable of ignoring technical reasons
	153	+in favor of political decisions.
	154	+
	155	+\textbf{L6 -- Consider sustainability from the beginning.}
	156	+In the process of deploying the SPB platform in the MP infrastructure we had
	157	+to interact with the MP technicians. We did several workshops, training
	158	+and a meticulous documentation describing all the required procedures to
	159	+update the platform, however, we realized that the MP technicians
	160	+constantly avoid the responsibility. After noticing the aforementioned
	161	+situation, we organized a DevOps team that completely automated all
	162	+the deployment procedure. We simplified all the platform deployment to a
	163	+few steps: (1) initial configurations (just ssh configuration) and (2)
	164	+the execution of simple commands to completely update the platform. By
	165	+the end of the project, we observed that the MP technicians invariably
	166	+still depended on our support to update the platform even with all the
	167	+automation provided by us. We were sadly left with a feeling of
	168	+uncertainty about the future of the platform after the project ended. In
	169	+hindsight, we realize that the MP dedicated system analysts and
	170	+managers to the project, but not operations technicians. The later
	171	+should have been more involved with the process so they could at least be
	172	+comfortable in managing the platform infrastructure.
	173	+
	174	+
	175	+\textbf{Final remarks and future work}
	176	+
	177	+The portal is available at \url{softwarepublico.gov.br}. All
	178	+documentation, including detailed architecture and operation manuals are
	179	+also available\footnote{\url{https://softwarepublico.gov.br/doc/}
	180	+(in Portuguese only at the moment)}.
	181	+%
	182	+All the integrated tools are FLOSS and our contributions were published
	183	+in open repositories, available on the SPB Portal itself. We also
	184	+contributed these features back to the respective communities, which
	185	+benefits both those communities and us, since we can share future
	186	+development and maintenance effort with other organizations that
	187	+participate in these projects.
	188	+
	189	+Future work should use data produced by the project to validate and evaluate
	190	+how the used FLOSS and Agile practices have impacted the students and the
	191	+governmental development process. For this, we would conduce a \textit{postmortem}
	192	+analysis using the project open data and a survey targeting the involved actors.
	193	+
	194	+%===========
	195	+% Conclusion
	196	+%===========
	197	+
	198	+% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
	199	+%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
	200	+%% afirmação, embora eu e Paulo consigamos perceber isso.
	201	+
	202	+
	203	+%* utilização do projeto para formação de recursos humanos (alunos)
	204	+
	205	+%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
	206	+
	207	+%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
	208	+
	209	+%* 69 projetos marcados como SPB, de 81 no total na plataforma.
	210	+
	211	+%* 47\% é desenvolvido em PHP.
	212	+
	213	+% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
	214	+
	215	+% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
	216	+
	217	+% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
	218	+
	219	+%* Debater economia de recursos em orgão públicos
	220	+
	@@ -1,121 +0,0 @@	@@ -1,121 +0,0 @@
1	-\section{Lessons Learned}
2	-\label{sec:lessons}
3	-
4	-\textbf{How to involve students in real-world projects, interacting with
5	-real customers.}
6	-%
7	-Our team was mainly composed of software engineering undergraduate
8	-students, who had the opportunity to interact with senior developers and
9	-designers on the team, as well as with the team of technicians and
10	-managers from the Brazilian Government, and the management team from
11	-UnB.
12	-%
13	-The students interacted with professionals of diverse fields of expertise, and they were
14	-able to participate in all levels of the software development process.
15	-This contributed to a great learning opportunity. Moreover, for the majority of
16	-the students, this was a first professional experience.
17	-
18	-\textbf{The participation of experienced professionals is crucial to
19	-the success of the project.}
20	-One important factor for the students was the composition of the teams
21	-with the participation of experienced professionals.
22	-%
23	-On the technical side, the senior developers and designers would handle
24	-the more difficult technical decisions, creating a work environment
25	-where the students could develop their skills in a didactic way without
26	-pressure.
27	-%
28	-On the management side, the active participation of professors -- who
29	-are, in the end, the ones responsible for the project -- is crucial to
30	-make sure students participation is conducted in a healthy way, and it is an
31	-instance of leading by example.
32	-
33	-\textbf{A balanced relationship between academia and industry.}
34	-The experience of the SPB project led UnB to develop a work style which
35	-proved to be appropriate for an educational environment that brings
36	-academia and industry together.
37	-%
38	-The highest priority from the university's point of view is the
39	-students. Considering this, the activities of the project were never
40	-prioritized to the detriment of classes and other pedagogical
41	-activities. In summary, we had students working at different times, part
42	-time, remotely or locally, always respecting their individual
43	-conditions, but doing the work in a collective, collaborative and open
44	-way.
45	-%
46	-And even under a potentially adverse environment, the project delivered
47	-the desired solution with success.
48	-%
49	-At the end of the project, we noted that the skills developed by the
50	-students were at the software engineering state of the art.
51	-After the project ended, we had team members successfully embracing
52	-opportunities in public, private, national, and international
53	-organizations, in addition to those students that
54	-opened their own companies.
55	-
56	-\textbf{Managing different organizational cultures.}
57	-In the beginning of the project, the Brazilian Government stakeholders
58	-had certain expectations about the project development that, let's
59	-say, didn't exactly match our work style based on agile and FLOSS practices.
60	-%
61	-We had to develop strategies that would support these different
62	-organizational cultures. As reported in Section \ref{sec:process}, the
63	-MP is organized in a functional hierarchical organizational structure,
64	-typically adopting a traditional development paradigm. Therefore, we had to
65	-create a translation process between our team and the MP managers who
66	-managed the project on their side assuming a traditional waterfall
67	-process.
68	-
69	-\textbf{Managing higher-level and lower-level goals separately.}
70	-During the initial phase of the project, the MP team has brought
71	-strategic discussions to technical/operational meetings that
72	-were supposed to be about practical technical decisions.
73	-%
74	-This produced a highly complex communication and management environment,
75	-overloading the professors because they were supposed
76	-for maintaining the MP strategy synchronized with the
77	-implementation plans of the development team. This was hard, especially because the
78	-aforementioned cultural mismatch. Mixing both concerns in the same
79	-discussions caused confusion on both sides.
80	-%
81	-From the middle of the project we were able to keep those
82	-concerns separated, what eased the work of everyone involved.
83	-
84	-\textbf{Living with ill-advised political decisions.}
85	-At the initial phases of the project, by political and personal
86	-motivation, the main stakeholders from the Brazilian government imposed
87	-the use of Colab to guide the development of the new SPB platform. Our
88	-team was totally against the idea because we already knew that Colab was
89	-a very experimental project and its adoption could dramatically increase
90	-the project complexity. Even though we provided technical reasons to
91	-not utilize Colab, the MP was adamant and we had to manage this problem. As
92	-explained in section \ref{sec:architecture}, we did massive changes to
93	-Colab, and at the end of the project we have completely rewritten it to make
94	-it stable. It is important to notice that the MP compelled us to accept
95	-a technical decision based only on political interests, without considering
96	-all the resources that would be spent due to that decision. At the end
97	-of the project, we verified that Colab indeed consumed a vast amount of
98	-the budget and increased the project complexity. At the end of the
99	-project and after our analysis on the decision made by the MP, we
100	-understand that MP managers are capable of ignoring technical reasons
101	-in favor of political decisions.
102	-
103	-\textbf{Consider sustainability from the beginning.}
104	-In the process of deploying the SPB platform in the MP infrastructure we had
105	-to interact with the MP technicians. We did several workshops, training
106	-and a meticulous documentation describing all the required procedures to
107	-update the platform, however, we realized that the MP technicians
108	-constantly avoid the responsibility. After noticing the aforementioned
109	-situation, we organized a DevOps team that completely automated all
110	-the deployment procedure. We simplified all the platform deployment to a
111	-few steps: (1) initial configurations (just ssh configuration) and (2)
112	-the execution of simple commands to completely update the platform. By
113	-the end of the project, we observed that the MP technicians invariably
114	-still depended on our support to update the platform even with all the
115	-automation provided by us. We were sadly left with a feeling of
116	-uncertainty about the future of the platform after the project ended. In
117	-hindsight, we realize that the MP dedicated system analysts and
118	-managers to the project, but not operations technicians. The later
119	-should have been more involved with the process so they could at least be
120	-comfortable in managing the platform infrastructure.
121	-
	@@ -1,95 +0,0 @@	@@ -1,95 +0,0 @@
1	-\section{Conclusion}
2	-\label{sec:conclusion}
3	-
4	-In this paper we presented and discussed issues experienced during a government-funded
5	-project, in partnership with the University of Brasilia and the University of
6	-São Paulo, to evolve the Brazilian Public Software Portal.
7	-Its contributions are twofold. First, we present the strategy used to develop
8	-and to deliver an unprecedented platform to Brazilian government. Second,
9	-based on the results of the SPB Portal project, we point out that it is
10	-possible to mitigate conflicts experienced in the development environment
11	-and to conciliate governmental and academy cultures.
12	-
13	-The SPB portal integrates more than 10 FLOSS tools and provides several features,
14	-such as social network, mailing list, version control, content management and
15	-source code quality monitoring. Concerned with the platform susteinability and
16	-maintainabilty, the aforementioned 10 FLOSS tools were integrated with minimum
17	-differences from their official versions and the new developed features were
18	-sent upstream to ensure an alignment between the portal systems and their
19	-respective official versions. In the integration process, the main softwares
20	-were identified, specific teams were formed to work with each one of them
21	-and each team was composed of students with different levels of skills and at
22	-least one senior professional.
23	-
24	-In terms of mitigating conflicts, we tried to show that, as long as the
25	-institution can provide a healthy and challenging environment to its students,
26	-one may conciliate studies and professional training in universities.
27	-In our work process, based on open and collaborative software development
28	-practices, students could negotiate their work schedule as well as count on IT
29	-professionals to solve development issues.
30	-Among the students, we have defined coachs for each team and a meta-coach
31	-(coach of whole project). All coaches, together with professors, have
32	-intermediated the comunication between client (Ministry of Planning of Brasil)
33	-and the rest of the group.
34	-After the end of the project, some students successfully
35	-embraced opportunities in public and private sectors, within national borders
36	-and abroad. Some other students went further and started their own companies.
37	-
38	-We also demonstrate that, with some adaptations/"translation processes", it is feasible
39	-to conciliate agile methodologies and FLOSS practices to develop software to
40	-governmental organizations with functional hierarchical structures that use
41	-traditional development paradigm.
42	-Aiming at reducing client questions about workconclusion, a DevOps front was
43	-created to automate all deploy process and also to work in continuous
44	-delivery. The government was brought to our work environment and interacted
45	-with our management and comunication tools. For the project success, we
46	-focused on providing a friendly working environment as well as on showing to
47	-governmental agents another way to interact with the FLOSS community and the
48	-university.
49	-
50	-Future work should use data produced by the project to validate and evaluate
51	-how the used FLOSS and Agile practices have impacted the students and the
52	-governmental development process. For this, we would conduce a \textit{postmortem}
53	-analysis using the project open data and a survey targeting the involved actors.
54	-
55	-\textbf{Final remarks}
56	-
57	-The portal is available at \url{softwarepublico.gov.br}. All
58	-documentation, including detailed architecture and operation manuals are
59	-also available\footnote{\url{https://softwarepublico.gov.br/doc/}
60	-(in Portuguese only at the moment)}.
61	-%
62	-All the integrated tools are FLOSS and our contributions were published
63	-in open repositories, available on the SPB Portal itself. We also
64	-contributed these features back to the respective communities, which
65	-benefits both those communities and us, since we can share future
66	-development and maintenance effort with other organizations that
67	-participate in these projects.
68	-
69	-%===========
70	-% Conclusion
71	-%===========
72	-
73	-% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados
74	-%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa
75	-%% afirmação, embora eu e Paulo consigamos perceber isso.
76	-
77	-
78	-%* utilização do projeto para formação de recursos humanos (alunos)
79	-
80	-%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate
81	-
82	-%* o que achamos que irá acontecer com o SPB no futuro breve (acabar)
83	-
84	-%* 69 projetos marcados como SPB, de 81 no total na plataforma.
85	-
86	-%* 47\% é desenvolvido em PHP.
87	-
88	-% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket).
89	-
90	-% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}.
91	-
92	-% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos.
93	-
94	-%* Debater economia de recursos em orgão públicos
95	-