From 5cef1922659e2e507dc9efd3cd73bbedb5718c91 Mon Sep 17 00:00:00 2001 From: Paulo Meirelles Date: Mon, 24 Jul 2017 16:41:04 -0300 Subject: [PATCH] Re-organizing the conclusion section --- opensym2017/content/06-architecture.tex | 2 +- opensym2017/content/11-conclusion.tex | 220 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ opensym2017/content/11-lessons.tex | 121 ------------------------------------------------------------------------------------------------------------------------- opensym2017/content/12-conclusion.tex | 95 ----------------------------------------------------------------------------------------------- opensym2017/spb.tex | 3 +-- 5 files changed, 222 insertions(+), 219 deletions(-) create mode 100644 opensym2017/content/11-conclusion.tex delete mode 100644 opensym2017/content/11-lessons.tex delete mode 100644 opensym2017/content/12-conclusion.tex diff --git a/opensym2017/content/06-architecture.tex b/opensym2017/content/06-architecture.tex index 1f414a5..ff7d066 100644 --- a/opensym2017/content/06-architecture.tex +++ b/opensym2017/content/06-architecture.tex @@ -131,7 +131,7 @@ as we can see in Figure \ref{fig:architecture2}. \begin{figure*}[hbt] \centering - \includegraphics[width=.8\linewidth]{figures/arch3.png} + \includegraphics[width=\linewidth]{figures/arch3.png} \caption{Instanciation view of the SPB architecture.} \label{fig:architecture2} \end{figure*} diff --git a/opensym2017/content/11-conclusion.tex b/opensym2017/content/11-conclusion.tex new file mode 100644 index 0000000..c1f181c --- /dev/null +++ b/opensym2017/content/11-conclusion.tex @@ -0,0 +1,220 @@ +\section{Conclusion} +\label{sec:conclusion} + +In this work, we presented and discussed issues experienced during a government-funded +project, in partnership with the University of Brasilia and the University of +São Paulo, to evolve the Brazilian Public Software Portal. +Its contributions are twofold. First, we present the strategy used to develop +and to deliver an unprecedented platform to Brazilian government. Second, +based on the results of the SPB Portal project, we point out that it is +possible to mitigate conflicts experienced in the development environment +and to conciliate governmental and academy cultures. To summarize our main contributions, we answered in this section the three open questions those guided this paper. + + +\textbf{Q1 -- Which strategy could be used to integrate several existing +FLOSS tools to promote the collaborative software development?} +% +The SPB portal integrates more than 10 FLOSS tools and provides several features, +such as social network, mailing list, version control, content management and +source code quality monitoring. Concerned with the platform sustainability and +maintainability, the aforementioned 10 FLOSS tools were integrated with minimum +differences from their official versions and the new developed features were +sent upstream to ensure an alignment between the portal systems and their +respective official versions. In the integration process, the main softwares +were identified, specific teams were formed to work with each one of them +and each team was composed of students with different levels of skills and at +least one senior professional. + +\textbf{Q2 -- How to involve students in real-world projects interacting with +real customers?} +% +In terms of mitigating conflicts, we tried to show that, as long as the +university can provide a healthy and challenging environment to its students, +one may conciliate studies and professional training in universities. +% +The students interacted with professionals of diverse fields of expertise, and they were +able to participate in all levels of the software development process. +This contributed to a great learning opportunity. +% +In our work process, based on open and collaborative software development +practices, students could negotiate their work schedule as well as count on IT +professionals to solve development issues. +% +Among the students, we have defined coaches for each team and a meta-coach +(coach of whole project). All coaches, together with professors, have +intermediated the communication between client (Ministry of Planning of Brazil) +and the rest of the group. +After the end of the project, some students successfully +embraced opportunities in public and private sectors, within national borders +and abroad. Some other students went further and started their own companies. + +\textbf{Q3 -- How to introduce the FLOSS collaborative and agile +practices to governmental development process?} +With some adaptations, what we called the ``translation processes'', it is feasible +to conciliate agile methodologies and FLOSS practices to develop software to +governmental organizations with functional hierarchical structures that use +traditional development paradigm. +% +Aiming at reducing client questions about workconclusion, a DevOps front was +created to automate all deploy process and also to work in continuous +delivery. The government was brought to our work environment and interacted +with our management and communication tools. For the project success, we +focused on providing a friendly working environment as well as on showing to +governmental agents another way to interact with the FLOSS community and the +university. + +\subsection{Lessons Learned} +\label{sec:lessons} + +From the answers of our initial open questions, we can also highlighted six lessons learned to make easier to share our experience during the development of the new SPB Portal. + +\textbf{L1 -- The participation of experienced professionals is crucial to +the success of the project.} +One important factor for the students was the composition of the teams +with the participation of experienced professionals. +% +On the technical side, the senior developers and designers would handle +the more difficult technical decisions, creating a work environment +where the students could develop their skills in a didactic way without +pressure. +% +On the management side, the active participation of professors -- who +are, in the end, the ones responsible for the project -- is crucial to +make sure students participation is conducted in a healthy way, and it is an +instance of leading by example. + +\textbf{L2 -- A balanced relationship between academia and industry.} +The experience of the SPB project led UnB to develop a work style which +proved to be appropriate for an educational environment that brings +academia and industry together. +% +The highest priority from the university's point of view is the +students. Considering this, the activities of the project were never +prioritized to the detriment of classes and other pedagogical +activities. In summary, we had students working at different times, part +time, remotely or locally, always respecting their individual +conditions, but doing the work in a collective, collaborative and open +way. +% +And even under a potentially adverse environment, the project delivered +the desired solution with success. +% +At the end of the project, we noted that the skills developed by the +students were at the software engineering state of the art. +After the project ended, we had team members successfully embracing +opportunities in public, private, national, and international +organizations, in addition to those students that +opened their own companies. + +\textbf{L3 -- Managing different organizational cultures.} +In the beginning of the project, the Brazilian Government stakeholders +had certain expectations about the project development that, let's +say, didn't exactly match our work style based on agile and FLOSS practices. +% +We had to develop strategies that would support these different +organizational cultures. The +MP is organized in a functional hierarchical organizational structure, +typically adopting a traditional development paradigm. Therefore, we had to +create a translation process between our team and the MP managers who +managed the project on their side assuming a traditional waterfall +process. + +\textbf{L4 -- Managing higher-level and lower-level goals separately.} +During the initial phase of the project, the MP team has brought +strategic discussions to technical/operational meetings that +were supposed to be about practical technical decisions. +% +This produced a highly complex communication and management environment, +overloading the professors because they were supposed +for maintaining the MP strategy synchronized with the +implementation plans of the development team. This was hard, especially because the +aforementioned cultural mismatch. Mixing both concerns in the same +discussions caused confusion on both sides. +% +From the middle of the project we were able to keep those +concerns separated, what eased the work of everyone involved. + +\textbf{L5 -- Living with ill-advised political decisions.} +At the initial phases of the project, by political and personal +motivation, the main stakeholders from the Brazilian government imposed +the use of Colab to guide the development of the new SPB platform. Our +team was totally against the idea because we already knew that Colab was +a very experimental project and its adoption could dramatically increase +the project complexity. Even though we provided technical reasons to +not utilize Colab, the MP was adamant and we had to manage this problem. We did massive changes to +Colab, and at the end of the project we have completely rewritten it to make +it stable. It is important to notice that the MP compelled us to accept +a technical decision based only on political interests, without considering +all the resources that would be spent due to that decision. At the end +of the project, we verified that Colab indeed consumed a vast amount of +the budget and increased the project complexity. At the end of the +project and after our analysis on the decision made by the MP, we +understand that MP managers are capable of ignoring technical reasons +in favor of political decisions. + +\textbf{L6 -- Consider sustainability from the beginning.} +In the process of deploying the SPB platform in the MP infrastructure we had +to interact with the MP technicians. We did several workshops, training +and a meticulous documentation describing all the required procedures to +update the platform, however, we realized that the MP technicians +constantly avoid the responsibility. After noticing the aforementioned +situation, we organized a DevOps team that completely automated all +the deployment procedure. We simplified all the platform deployment to a +few steps: (1) initial configurations (just ssh configuration) and (2) +the execution of simple commands to completely update the platform. By +the end of the project, we observed that the MP technicians invariably +still depended on our support to update the platform even with all the +automation provided by us. We were sadly left with a feeling of +uncertainty about the future of the platform after the project ended. In +hindsight, we realize that the MP dedicated system analysts and +managers to the project, but not operations technicians. The later +should have been more involved with the process so they could at least be +comfortable in managing the platform infrastructure. + + +\textbf{Final remarks and future work} + +The portal is available at \url{softwarepublico.gov.br}. All +documentation, including detailed architecture and operation manuals are +also available\footnote{\url{https://softwarepublico.gov.br/doc/} +(in Portuguese only at the moment)}. +% +All the integrated tools are FLOSS and our contributions were published +in open repositories, available on the SPB Portal itself. We also +contributed these features back to the respective communities, which +benefits both those communities and us, since we can share future +development and maintenance effort with other organizations that +participate in these projects. + +Future work should use data produced by the project to validate and evaluate +how the used FLOSS and Agile practices have impacted the students and the +governmental development process. For this, we would conduce a \textit{postmortem} +analysis using the project open data and a survey targeting the involved actors. + +%=========== +% Conclusion +%=========== + +% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados +%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa +%% afirmação, embora eu e Paulo consigamos perceber isso. + + +%* utilização do projeto para formação de recursos humanos (alunos) + +%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate + +%* o que achamos que irá acontecer com o SPB no futuro breve (acabar) + +%* 69 projetos marcados como SPB, de 81 no total na plataforma. + +%* 47\% é desenvolvido em PHP. + +% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket). + +% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}. + +% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos. + +%* Debater economia de recursos em orgão públicos + diff --git a/opensym2017/content/11-lessons.tex b/opensym2017/content/11-lessons.tex deleted file mode 100644 index bde54e9..0000000 --- a/opensym2017/content/11-lessons.tex +++ /dev/null @@ -1,121 +0,0 @@ -\section{Lessons Learned} -\label{sec:lessons} - -\textbf{How to involve students in real-world projects, interacting with -real customers.} -% -Our team was mainly composed of software engineering undergraduate -students, who had the opportunity to interact with senior developers and -designers on the team, as well as with the team of technicians and -managers from the Brazilian Government, and the management team from -UnB. -% -The students interacted with professionals of diverse fields of expertise, and they were -able to participate in all levels of the software development process. -This contributed to a great learning opportunity. Moreover, for the majority of -the students, this was a first professional experience. - -\textbf{The participation of experienced professionals is crucial to -the success of the project.} -One important factor for the students was the composition of the teams -with the participation of experienced professionals. -% -On the technical side, the senior developers and designers would handle -the more difficult technical decisions, creating a work environment -where the students could develop their skills in a didactic way without -pressure. -% -On the management side, the active participation of professors -- who -are, in the end, the ones responsible for the project -- is crucial to -make sure students participation is conducted in a healthy way, and it is an -instance of leading by example. - -\textbf{A balanced relationship between academia and industry.} -The experience of the SPB project led UnB to develop a work style which -proved to be appropriate for an educational environment that brings -academia and industry together. -% -The highest priority from the university's point of view is the -students. Considering this, the activities of the project were never -prioritized to the detriment of classes and other pedagogical -activities. In summary, we had students working at different times, part -time, remotely or locally, always respecting their individual -conditions, but doing the work in a collective, collaborative and open -way. -% -And even under a potentially adverse environment, the project delivered -the desired solution with success. -% -At the end of the project, we noted that the skills developed by the -students were at the software engineering state of the art. -After the project ended, we had team members successfully embracing -opportunities in public, private, national, and international -organizations, in addition to those students that -opened their own companies. - -\textbf{Managing different organizational cultures.} -In the beginning of the project, the Brazilian Government stakeholders -had certain expectations about the project development that, let's -say, didn't exactly match our work style based on agile and FLOSS practices. -% -We had to develop strategies that would support these different -organizational cultures. As reported in Section \ref{sec:process}, the -MP is organized in a functional hierarchical organizational structure, -typically adopting a traditional development paradigm. Therefore, we had to -create a translation process between our team and the MP managers who -managed the project on their side assuming a traditional waterfall -process. - -\textbf{Managing higher-level and lower-level goals separately.} -During the initial phase of the project, the MP team has brought -strategic discussions to technical/operational meetings that -were supposed to be about practical technical decisions. -% -This produced a highly complex communication and management environment, -overloading the professors because they were supposed -for maintaining the MP strategy synchronized with the -implementation plans of the development team. This was hard, especially because the -aforementioned cultural mismatch. Mixing both concerns in the same -discussions caused confusion on both sides. -% -From the middle of the project we were able to keep those -concerns separated, what eased the work of everyone involved. - -\textbf{Living with ill-advised political decisions.} -At the initial phases of the project, by political and personal -motivation, the main stakeholders from the Brazilian government imposed -the use of Colab to guide the development of the new SPB platform. Our -team was totally against the idea because we already knew that Colab was -a very experimental project and its adoption could dramatically increase -the project complexity. Even though we provided technical reasons to -not utilize Colab, the MP was adamant and we had to manage this problem. As -explained in section \ref{sec:architecture}, we did massive changes to -Colab, and at the end of the project we have completely rewritten it to make -it stable. It is important to notice that the MP compelled us to accept -a technical decision based only on political interests, without considering -all the resources that would be spent due to that decision. At the end -of the project, we verified that Colab indeed consumed a vast amount of -the budget and increased the project complexity. At the end of the -project and after our analysis on the decision made by the MP, we -understand that MP managers are capable of ignoring technical reasons -in favor of political decisions. - -\textbf{Consider sustainability from the beginning.} -In the process of deploying the SPB platform in the MP infrastructure we had -to interact with the MP technicians. We did several workshops, training -and a meticulous documentation describing all the required procedures to -update the platform, however, we realized that the MP technicians -constantly avoid the responsibility. After noticing the aforementioned -situation, we organized a DevOps team that completely automated all -the deployment procedure. We simplified all the platform deployment to a -few steps: (1) initial configurations (just ssh configuration) and (2) -the execution of simple commands to completely update the platform. By -the end of the project, we observed that the MP technicians invariably -still depended on our support to update the platform even with all the -automation provided by us. We were sadly left with a feeling of -uncertainty about the future of the platform after the project ended. In -hindsight, we realize that the MP dedicated system analysts and -managers to the project, but not operations technicians. The later -should have been more involved with the process so they could at least be -comfortable in managing the platform infrastructure. - diff --git a/opensym2017/content/12-conclusion.tex b/opensym2017/content/12-conclusion.tex deleted file mode 100644 index 3d76865..0000000 --- a/opensym2017/content/12-conclusion.tex +++ /dev/null @@ -1,95 +0,0 @@ -\section{Conclusion} -\label{sec:conclusion} - -In this paper we presented and discussed issues experienced during a government-funded -project, in partnership with the University of Brasilia and the University of -São Paulo, to evolve the Brazilian Public Software Portal. -Its contributions are twofold. First, we present the strategy used to develop -and to deliver an unprecedented platform to Brazilian government. Second, -based on the results of the SPB Portal project, we point out that it is -possible to mitigate conflicts experienced in the development environment -and to conciliate governmental and academy cultures. - -The SPB portal integrates more than 10 FLOSS tools and provides several features, -such as social network, mailing list, version control, content management and -source code quality monitoring. Concerned with the platform susteinability and -maintainabilty, the aforementioned 10 FLOSS tools were integrated with minimum -differences from their official versions and the new developed features were -sent upstream to ensure an alignment between the portal systems and their -respective official versions. In the integration process, the main softwares -were identified, specific teams were formed to work with each one of them -and each team was composed of students with different levels of skills and at -least one senior professional. - -In terms of mitigating conflicts, we tried to show that, as long as the -institution can provide a healthy and challenging environment to its students, -one may conciliate studies and professional training in universities. -In our work process, based on open and collaborative software development -practices, students could negotiate their work schedule as well as count on IT -professionals to solve development issues. -Among the students, we have defined coachs for each team and a meta-coach -(coach of whole project). All coaches, together with professors, have -intermediated the comunication between client (Ministry of Planning of Brasil) -and the rest of the group. -After the end of the project, some students successfully -embraced opportunities in public and private sectors, within national borders -and abroad. Some other students went further and started their own companies. - -We also demonstrate that, with some adaptations/"translation processes", it is feasible -to conciliate agile methodologies and FLOSS practices to develop software to -governmental organizations with functional hierarchical structures that use -traditional development paradigm. -Aiming at reducing client questions about workconclusion, a DevOps front was -created to automate all deploy process and also to work in continuous -delivery. The government was brought to our work environment and interacted -with our management and comunication tools. For the project success, we -focused on providing a friendly working environment as well as on showing to -governmental agents another way to interact with the FLOSS community and the -university. - -Future work should use data produced by the project to validate and evaluate -how the used FLOSS and Agile practices have impacted the students and the -governmental development process. For this, we would conduce a \textit{postmortem} -analysis using the project open data and a survey targeting the involved actors. - -\textbf{Final remarks} - -The portal is available at \url{softwarepublico.gov.br}. All -documentation, including detailed architecture and operation manuals are -also available\footnote{\url{https://softwarepublico.gov.br/doc/} -(in Portuguese only at the moment)}. -% -All the integrated tools are FLOSS and our contributions were published -in open repositories, available on the SPB Portal itself. We also -contributed these features back to the respective communities, which -benefits both those communities and us, since we can share future -development and maintenance effort with other organizations that -participate in these projects. - -%=========== -% Conclusion -%=========== - -% * Gestão dos recursos: Fizemos mais por menos (2.6M de 3.2M) --- sem os dados -%% (escopo, custo, tempo e qualidade) bem discutidos é difícil sustentar essa -%% afirmação, embora eu e Paulo consigamos perceber isso. - - -%* utilização do projeto para formação de recursos humanos (alunos) - -%* dados da verificação dos repositório para a análise da qualidade dos código via Mezuro e CodeClimate - -%* o que achamos que irá acontecer com o SPB no futuro breve (acabar) - -%* 69 projetos marcados como SPB, de 81 no total na plataforma. - -%* 47\% é desenvolvido em PHP. - -% foi constatado que aproximadamente 75\% dos softwares \textbf{não} possuem seus códigos-fonte versionados nesta ferramenta. Realizado algumas pesquisas, foi encontrado o código-fonte em outros serviços (Github, Bitbucket). - -% Foram adicionados 31 softwares do SPB em ambas as ferramentas (Mezuro e Code Climate), desenvolvidos em PHP e Python. Estas adições resultaram na análise descrita nos próximos parágrafos. No Mezuro, dos 31 softwares adicionados, somente 4 obtiveram sucesso na avaliação. No Code Climate, 16 softwares realizaram a \textit{build} da avaliação com sucesso. Nos que falharam, alguns dos erros foram encontrados em três das \textit{engines}: ora em \textit{duplication}, ora na \textit{phpmd}, ora na \textit{eslint}. - -% também foram inseridos no Mezuro para avaliação, 5 projetos dos 17 desenvolvidos em Java, com o intuito de ser um contraponto ao Code Climatepor esta não compreender a análise de projetos em Java, C, ou C++. Infelizmente nenhuma das \textit{builds} resultou em resultados concretos. - -%* Debater economia de recursos em orgão públicos - diff --git a/opensym2017/spb.tex b/opensym2017/spb.tex index 210986e..1ba9ea9 100644 --- a/opensym2017/spb.tex +++ b/opensym2017/spb.tex @@ -187,8 +187,7 @@ \input{content/08-ux} \input{content/09-process} \input{content/10-contributions} -\input{content/11-lessons} -\input{content/12-conclusion} +\input{content/11-conclusion} %------------------------------------------------------------------------------ \bibliographystyle{SIGCHI-Reference-Format} -- libgit2 0.21.2