Merge branch 'master' of softwarepublico.gov.br:softwarepublico/articles

Conflicts: opensym2017/spb.bib

Merge branch 'master' of softwarepublico.gov.br:softwarepublico/articles
Conflicts: opensym2017/spb.bib
Paulo Meireles
2 parents 29d3dfd4 9ab72412
Showing 6 changed files with 181 additions and 146 deletions Show diff stats
opensym2017/content/00-abstract.tex
opensym2017/content/01-introduction.tex
opensym2017/content/11-lessons.tex
opensym2017/content/12-conclusion.tex
opensym2017/spb.bib
opensym2017/spb.tex
 \begin{abstract}
-The Brazilian Public Software (SPB) is a program by the Brazilian Federal
-Government to foster the sharing and collaboration on Free/Libre/Open Source
-Software (FLOSS) solutions for the public administration. In the one hand,
-Brazilian Public Softwares have some differences from FLOSS projects, in
-particular, the software is considered a public good and the Federal government
-assumes some responsibilities related to its use. In the other hand, the
+The Brazilian Public Software (SPB) is a program promoted by the Brazilian Federal
+Government to foster sharing and collaboration on Free/Libre/Open Source
+Software (FLOSS) solutions for the public administration. On the one hand,
+Brazilian Public Software has some differences from other FLOSS projects. In
+particular, the public software is considered a public good and the Federal Government
+assumes some responsibilities related to its use. On the other hand, the
 software development principles are the same: the trend towards
-decentralization in decision-making, the information and development sharing,
-and the continuous feedback. In this context, we have designed a platform based
-on the integration and evolution of existing FLOSS tools. Nowadays, the SPB
-Portal provides several modern features for software collaborative development,
-helping the Brazilian public administration to share its solutions. In this
+decentralization in decision-making, information and code sharing,
+and continuous feedback. In this context, we have designed the SPB Portal, a platform based
+on the integration and evolution of existing FLOSS tools. Nowadays, it
+provides several modern features for software collaborative development,
+helping the Brazilian public administration in sharing its solutions. In this
 paper, we present this integrated software development platform that was
-developed for the program by a heterogeneous team composed by professors,
-master students, undergraduate students and professionals from FLOSS
+developed for the SPB program by a heterogeneous team composed by professors,
+master students, undergraduate students, and professionals from FLOSS
 communities. The development of this platform used several FLOSS applications,
-providing a non-trivial integration among them, as well as, have generated
-several contributed features back to them. In this experience report, alongside
-the platform architecture, features, and the user experience efforts carried
-out, we also discuss our work process, based on agile and free software
-development practices, and the lessons learned in 30 months work on the SPB
+providing a non-trivial integration among them. This effort has also produced
+several new features that were contributed back to these FLOSS applications. In this experience report, alongside
+the platform architecture, the features, and the user experience challenges,
+we also discuss our work process, based on agile and free software
+development practices, and the lessons learned in 30 months of work on the SPB
 project.
 \end{abstract} 
@@ -2,49 +2,50 @@
 \label{sec:intro}
 The Brazilian Federal Government has been 
-improving its software adoption and development processes. For
+improving its processes for software contracting and development. For
 instance, in 2003, the recommendation to adopt Free/Libre/Open Source
-Software (FLOSS) become a public policy. In 2007, the Brazilian
-Government released a portal named Brazilian Public Software
-(\textit{Software Público Brasileiro} -- SPB, in Portuguese), with the
+Software (FLOSS) became a public policy. In 2007, the Brazilian
+Government released the Brazilian Public Software Portal
+(\textit{Portal do Software Público Brasileiro}, in Portuguese), with the
 goal of sharing FLOSS projects developed by, or for, the Brazilian
 Government.  Additionally, the Brazilian legal instrument on software
-contracting (known as IN 04/2012) mandates that public agents must give
-priority to solutions available on the SPB Portal.  In short, the
+contracting (known as IN 04/2012) mandates that public agents must 
+prioritize solutions available on the SPB Portal. The
 acquisition of a proprietary solution must be explicitly justified by
 demonstrating that there is no suitable alternative on the SPB Portal.
-In 2013, the Brazilian Federal Court issued a ruling document
-(\textit{Acórdão 2314/2013}) about an audit survey regarding the use of
-agile methodologies in software development contracts with the public
-administration.
+In 2013, the Brazilian Federal Court issued a ruling
+(\textit{Acórdão 2314/2013}) about contracts between the public administration
+and suppliers using agile methodologies in software development.
-Despite of that, in practice, free software or agile methodologies, that is,
-collaborative and empirical software development methods are not widely
-practiced and understood by the Brazilian government agents. Thus, the
-hierarchical and traditional processes from the government and the lack
-of expertise in real-world software development of its agents produces a
-situation of inefficient software development contracts and
-unjustifiable expending of taxpayers' money.
+Despite of these legal advancements, in practice, Brazilian government agents
+still do not practice, or even understand, 
+collaborative and empirical software development methods,
+such as free software or agile methodologies. Thus,
+hierarchical and traditional processes and the lack
+of expertise of public agents in real-world software development produce
+inefficiency in software development contracts, besides
+unjustifiable expending of taxpayers money.
-Since 2009, the SPB Portal was having several technical issues. The original
-codebase was not being developed anymore, also, there was a large amount of
-knowingly non-optimal or wrong design decisions to overcome (in other words,
-technical debt \cite{refactoring}). The system was a modified version of an
+Since 2009, the SPB Portal has had several technical issues. The original
+codebase development has stopped, leaving substantial
+technical debt~\cite{refactoring}.
+The system was a modified version of an
 existing FLOSS platform called OpenACS \footnote{\url{http://openacs.org}}, and
-the old SPB portal was not being updated anymore against the official OpenACS
-releases. In this scenario, the portal maintenance has become increasingly
+the old SPB Portal did not receive updates from OpenACS releases.
+In this scenario, the portal maintenance has become increasingly
 difficult.
-After some events and meetings to collect requirements from the federal
-government and from the society, a new platform for the SPB Portal was
-developed, among January 2014 and June 2016, by the University of Brasília
+After collecting requirements from the Federal
+Government and society, a new platform for the SPB Portal was
+developed, between January 2014 and June 2016, by the University of Brasília
 (UnB) and the University of São Paulo (USP) in a partnership with the Brazilian
-Ministry of Budget, Planning, and Management (MP). It was designed as an
-integrated platform for collaborative software development \cite{bobr2003}, and
-includes functionality for social networking, mailing lists, version control
+Ministry of Planning, Budget, and Management (MP). It was designed as an
+integrated platform for collaborative software development \cite{bobr2003},
+including social networking, mailing lists, version control
 system, and source code quality monitoring. To coordinate and develop this
-project during 30 months, UnB received from the Brazilian Federal Government a
-total of 2,619,965.00 BRL (about 750,000.00 USD in June 2016).
+project during 30 months, UnB was funded by a grant
+of 2,619,965.00 BRL (about 750,000.00 USD in June 2016)
+from the Federal Government.
 \begin{figure*}[hbt]
   \centering
@@ -53,44 +54,45 @@ total of 2,619,965.00 BRL (about 750,000.00 USD in June 2016).
   \label{fig:spb}
 \end{figure*}
-The project was developed by a team of 3 professors, 2 masters students, and
-approximately 50 undergraduate students (not all of them at the same time,
-though -- graduations and other events triggered changes in the team) together
-with 2 professional designers and 6 senior developers from free software
-communities. The professors and all undergraduate student were from UnB, and
-the master students were from USP. Regarding the designers and senior
-developers, 7 of 8 they were living outside of Brasília: Curitiba/Brazil, São
-Paulo/Brazil, Ribeirão Preto/Brazil, Salvador/Brazil, Santo Domingo/Dominican
-Republic, and Montreal/Canada. In other words, we had a team working in
-distributed collaborative virtual environment. This diversity of actors and the
-relationships between industry, academy and government also made the project a
+The project was developed by a team of three professors, two masters students,
+about fifty undergraduate students (not all of them at the same time,
+since the team changed along the time),
+two professional designers, and six senior developers from free software
+communities. Professors and undergraduate students were from UnB and
+master students were from USP. Regarding the designers and senior
+developers, seven of eight were living outside Brasília, the UnB location.
+Two of them were abroad at Dominican Republic and Canada.
+In other words, we had a distributed team working in a
+collaborative virtual environment. This diversity of actors and the
+relationships between industry, academy, and government also made the project a
 valued opportunity to explore the benefits and challenges of using
-FLOSS\cite{kon2011,deKoenigsberg2008, fagerholm2013, fagerholm2014} and
-Agile\cite{steghofer2016, harzl2017} practices for Software Engineering
+FLOSS~\cite{kon2011,deKoenigsberg2008, fagerholm2013, fagerholm2014} and
+Agile~\cite{steghofer2016, harzl2017} practices for Software Engineering
 education.
 Figure \ref{fig:spb} shows the home page of this integrated platform.
-All development was done in the open, and the changes we needed in the
-FLOSS tools were contributed back to their respective communities. Our
+All the code was developed as open source. The changes we needed in the
+FLOSS tools were implemented by ourselves and
+contributed back to their respective communities. Our
 process was based on agile practices and FLOSS communities interaction.
-We defined development cycles and released 5 versions of the new SPB
+We incrementally released five versions of the new SPB
 Portal. The first release (beta) was in September 2014, only 9 months
 from the beginning of the project. The old portal was shut down in
 September 2015. Finally, the last version, illustrated in Figure 1, was
 released in June 2016.
-In this paper, we present an overview of this new generation of the SPB Portal.
-The paper shares the methodology employed to develop this project, in
-partnership with the Brazilian Federal Government, to comply with its
-requirements at the same time to be as faithful as possible to FLOSS
-development \cite{mockus2002, tosi2015}. Moreover, we discuss several lessons
-learned to provide a distributed collaborative virtual environment involving
-alarge undergraduate student team and remote senior developers. Lastly, we
-released an unprecedented platform for the Brazilian government applying
+In this paper, we present an overview of the new SPB Portal.
+The paper shares the methodology employed to develop this project.
+This methodology has the goals of satisfying Government 
+requirements and adhering as much as possible to FLOSS and agile
+practices~\cite{mockus2002, tosi2015}. Moreover, we discuss lessons
+learned in providing a distributed and collaborative virtual environment involving
+a large undergraduate students team and remote senior developers. Finally, we
+released an innovative platform for helping the Brazilian government to apply
 empirical software development methods. This case can help other projects to
 overcome similar software engineering challenges in the future, as well as to
 illustrate how universities can improve the real-world experience of their
-students by means of this kind of project.
+students.
 The remainder of this work is organized as follows.
 Section \ref{sec:spb}...
 \section{Lessons Learned}
 \label{sec:lessons}
-\textbf{How to involve students real-world projects, interacting with
+\textbf{How to involve students in real-world projects, interacting with
 real customers.}
 %
 Our team was mainly composed of software engineering undergraduate
-student, who had the opportunity to interact with senior developers and
-designers inside the team, as well as with the team of technicians and
+students, who had the opportunity to interact with senior developers and
+designers on the team, as well as with the team of technicians and
 managers from the Brazilian Government, and the management team from
 UnB.
 %
-They interacted with professionals that had diverse expertises, and were
+The students interacted with professionals of diverse fields of expertise, and they were
 able to participate in all levels of the software development process.
-This contributed to a great learning opportunity, and for a majority of
-them this was their first professional experience.
+This contributed to a great learning opportunity. Moreover, for the majority of
+the students, this was a first professional experience.
 \textbf{The participation of experienced professionals is crucial to
-success of the project.}
+the success of the project.}
 One important factor for the students was the composition of the teams
 with the participation of experienced professionals.
 %
@@ -27,7 +27,7 @@ pressure.
 %
 On the management side, the active participation of professors -- who
 are, in the end, the ones responsible for the project -- is crucial to
-make sure student participation is conducted in a healthy way, and is an
+make sure students participation is conducted in a healthy way, and it is an
 instance of leading by example.
 \textbf{A balanced relationship between academia and industry.}
@@ -47,66 +47,66 @@ And even under a potentially adverse environment, the project delivered
 the desired solution with success.
 %
 At the end of the project, we noted that the skills developed by the
-students were at the state of art of the software engineering practice.
+students were at the software engineering state of the art.
 After the project ended, we had team members successfully embracing
-opportunities in public, private, national and international
-organizations, in addition to those students that went into
-entrepreneurship and opened their own companies.
+opportunities in public, private, national, and international
+organizations, in addition to those students that
+opened their own companies.
 \textbf{Managing different organizational cultures.}
 In the beginning of the project, the Brazilian Government stakeholders
-had certain expectations about the development of project that, let's
-say, didn't exactly match our work based on agile and FOSS practices.
+had certain expectations about the project development that, let's
+say, didn't exactly match our work style based on agile and FOSS practices.
 %
-We had to develop strategies that would support different these
+We had to develop strategies that would support these different
 organizational cultures. As reported in Section \ref{sec:process}, the
 MP is organized in a functional hierarchical organizational structure,
-typically, a  traditional development paradigm. Therefore, we had to
+typically adopting a traditional development paradigm. Therefore, we had to
 create a translation process between our team and the MP managers who
-managed the project on their side assuming a traditional, waterfall
+managed the project on their side assuming a traditional waterfall
 process.
-\textbf{Manage higher-level and lower-level goals separately.}
-During the initial phase of the project the MP team would often bring
-strategic discussions to technical/operational meetings, where we were
-supposed to discuss practical, technical decisions.
+\textbf{Managing higher-level and lower-level goals separately.}
+During the initial phase of the project, the MP team has brought
+strategic discussions to technical/operational meetings that
+were supposed to be about practical technical decisions.
 %
 This produced a highly complex communication and management environment,
-overloading the professors because they were supposed to be responsible
-for maintaining the strategic alignment of the MP synchronized with
-implementation plans of the development team, specially in light of the
-aforementioned culture mismatch. Mixing both concerns in the same
+overloading the professors because they were supposed 
+for maintaining the MP strategy synchronized with the
+implementation plans of the development team. This was hard, especially because the
+aforementioned cultural mismatch. Mixing both concerns in the same
 discussions caused confusion on both sides.
 %
-Towards the middle and end of the project we were able to keep those
+From the middle of the project we were able to keep those
 concerns separated, what eased the work of everyone involved.
 \textbf{Living with ill-advised political decisions.}
 At the initial phases of the project, by political and personal
 motivation, the main stakeholders from the Brazilian government imposed
-the use of Colab to guide the development of the new SPB platform.  Our
+the use of Colab to guide the development of the new SPB platform. Our
 team was totally against the idea because we already knew that Colab was
 a very experimental project and its adoption could dramatically increase
-the project complexity.  Even though we provided technical reasons to
-not utilize Colab, MP was adamant and we had to manage this problem. As
-explained in section \ref{sec:architecture} we did massive changes to
-it, and at the end of the project we completely rewrote Colab and made
+the project complexity. Even though we provided technical reasons to
+not utilize Colab, the MP was adamant and we had to manage this problem. As
+explained in section \ref{sec:architecture}, we did massive changes to
+Colab, and at the end of the project we have completely rewritten it to make
 it stable. It is important to notice that the MP compelled us to accept
-a technical issue based only on political interests, without considering
+a technical decision based only on political interests, without considering
 all the resources that would be spent due to that decision. At the end
 of the project, we verified that Colab indeed consumed a vast amount of
-the budget and increased the project complexity. In the end of the
+the budget and increased the project complexity. At the end of the
 project and after our analysis on the decision made by the MP, we
-understand that MP's managers are capable of ignoring technical reasons
-in favor to a political decision.
+understand that MP managers are capable of ignoring technical reasons
+in favor of political decisions.
 \textbf{Consider sustainability from the beginning.}
-In the process of deploying the SPB platform in the MP structure, we had
+In the process of deploying the SPB platform in the MP infrastructure we had
 to interact with the MP technicians. We did several workshops, training
 and a meticulous documentation describing all the required procedures to
 update the platform, however, we realized that the MP technicians
 constantly avoid the responsibility. After noticing the aforementioned
-situation, we organized a specific DevOps that completely automated all
+situation, we organized a DevOps team that completely automated all
 the deployment procedure. We simplified all the platform deployment to a
 few steps: (1) initial configurations (just ssh configuration) and (2)
 the execution of simple commands to completely update the platform. By
@@ -114,8 +114,8 @@ the end of the project, we observed that the MP technicians invariably
 still depended on our support to update the platform even with all the
 automation provided by us. We were sadly left with a feeling of
 uncertainty about the future of the platform after the project ended. In
-hindsight, we realize that the MP dedicated systems analysts and
+hindsight, we realize that the MP dedicated system analysts and
 managers to the project, but not operations technicians. The later
-should have been more involved with the process so they could at least
-comfortable with managing the platform infrastructure.
+should have been more involved with the process so they could at least be
+comfortable in managing the platform infrastructure.
 \section{Conclusion}
 \label{sec:conclusion}
-In this paper we present and discuss issues experienced during a government-
-funded project, in partnership with University of Brasilia and University of
-São Paulo, to evolve the Brazilian Public Software portal. 
-
-The contributions of this paper are twofold. First, we present how an unprecedent
-platform was developed and delivered to Brazilian government. This 
-platform - developed by an heterogenous team of professors, masters and 
-undergraduate  students, IT professionals and governmental managers - provides 
-several modern features from the integration of more than 10 FLOSS systems.
-
-Second, the thirty-month long project in which the SPB platform was developed points
-out that it is possible to mitigate issues seen as conflicting to the IT
-development environment and between industry and academy. The team was engaged in
-providing a friendly working environment as well as in showing to governmental agents
-another way to interact with the FLOSS community and the university. In the paper,
-we try to show that, as long as the institution can provide a healthy and challenging
-environment to its students, one may conciliate studies and professional training
-in universities. After the end of the project, some students successfully
-embraced opportunities in public and private sectos, within national borders
-and abroad. Some others went further and started their own companies.
+In this paper we presented and discussed issues experienced during a government-funded 
+project, in partnership with the University of Brasilia and the University of
+São Paulo, to evolve the Brazilian Public Software Portal. 
+Its contributions are twofold. First, we present the strategy used to develop
+and to deliver an unprecedented platform to Brazilian government. Second,
+based on the results of the SPB Portal project, we point out that it is
+possible to mitigate conflicts experienced in the development environment
+and to conciliate governmental and academy cultures. 
+
+The SPB portal integrates more than 10 FOSS tools and provides several features,
+such as social network, mailing list, version control, content management and
+source code quality monitoring. Concerned with the platform susteinability and
+maintainabilty, the aforementioned 10 FOSS tools were integrated with minimum
+differences from their official versions and the new developed features were
+sent upstream to ensure an alignment between the portal systems and their
+respective official versions. In the integration process, the main softwares
+were identified, specific teams were formed to work with each one of them
+and each team was composed of students with different levels of skills and at
+least one senior professional.
+
+In terms of mitigating conflicts, we tried to show that, as long as the
+institution can provide a healthy and challenging environment to its students,
+one may conciliate studies and professional training in universities.
+In our work process, based on open and collaborative software development
+practices, students could negotiate their work schedule as well as count on IT
+professionals to solve development issues.
+Among the students, we have defined coachs for each team and a meta-coach
+(coach of whole project). All coaches, together with professors, have
+intermediated the comunication between client (Ministry of Planning of Brasil)
+and the rest of the group. 
+After the end of the project, some students successfully
+embraced opportunities in public and private sectors, within national borders
+and abroad. Some other students went further and started their own companies.
+
 We also demonstrate that, with some adaptations/"translation processes", it is feasible
-to conciliate agile methodologies and FOSS practices in order to develop software to 
+to conciliate agile methodologies and FOSS practices to develop software to 
 governmental organizations with functional hierarchical structures that use 
 traditional development paradigm.
+Aiming at reducing client questions about workconclusion, a DevOps front was
+created to automate all deploy process and also  to work in continuous
+delivery. The government was brought to our work environment and interacted
+with our management and comunication tools. For the project success, we
+focused on providing a friendly working environment as well as on showing to
+governmental agents another way to interact with the FLOSS community and the
+university. 
+
+\leo{Padronziar uso FOSS vs FLOSS. Qual vcs preferem?}
-Future work should use the many data produced by the project to validate and evaluate
-how the used FLOSS and Agile practices have impacted the students and also the
-governmental development process. For this, we would conduce a \textit{post-mortem}
-analyse using the project open data and a survey targeted the involved actors.
+Future work should use data produced by the project to validate and evaluate
+how the used FOSS and Agile practices have impacted the students and the
+governmental development process. For this, we would conduce a \textit{postmortem}
+analysis using the project open data and a survey targeting the involved actors.
+\textbf{Final remarks}
 The portal is available at \url{softwarepublico.gov.br}. All
 documentation, including detailed architecture and operation manuals are
 also available\footnote{\url{https://softwarepublico.gov.br/doc/}
-(in Portuguese only at the moment)}).
+(in Portuguese only at the moment)}.
 %
 All the integrated tools are FOSS and our contributions were published
 in open repositories, available on the SPB Portal itself. We also
-contributed these features back to the respective communities: that
-benefits those communities, as well as us since we can share future
+contributed these features back to the respective communities, which
+benefits both those communities and us, since we can share future
 development and maintenance effort with other organizations that
-participate in their projects.
+participate in these projects. 
 %===========
 % Conclusion
@@ -189,7 +189,8 @@
   doi       = {10.1016/S0065-2458(03)59001-5},
   timestamp = {Sat, 20 May 2017 00:22:37 +0200},
   biburl    = {http://dblp.uni-trier.de/rec/bib/journals/ac/BoochB03},
-  bibsource = {dblp computer science bibliography, http://dblp.org}
+  bibsource = {dblp computer science bibliography, http://dblp.org},
+  publisher={Elsevier}
 }
 @inproceedings{chao2007student,
@@ -199,8 +200,7 @@
   pages={255--261},
   year={2007},
   url = {http://dblp.uni-trier.de/db/conf/csee/csee2007.html},
-  publisher = {IEEE Computer Society},
-  organization={IEEE}
+  publisher = {IEEE Computer Society}
 }
 @inproceedings{tosi2015,
@@ -72,6 +72,13 @@
 % by using the "\todo{...}"  command. Make sure to disable the draft
 % option again before submitting your final document.
 \usepackage{todonotes}
+\newcommand{\leo}[2][]{\todo[inline,color=yellow!70, #1]{\emph{Léo:} #2}}
+\newcommand{\paulo}[2][]{\todo[inline,color=yellow!70, #1]{\emph{Paulo:} #2}}
+\newcommand{\melissa}[2][]{\todo[inline,color=yellow!70, #1]{\emph{Melissa:} #2}}
+\newcommand{\terceiro}[2][]{\todo[inline,color=yellow!70, #1]{\emph{Terceiro:} #2}}
+\newcommand{\siqueira}[2][]{\todo[inline,color=yellow!70, #1]{\emph{Siqueira:} #2}}
+\newcommand{\lucas}[2][]{\todo[inline,color=yellow!70, #1]{\emph{Lucas:} #2}}
+\newcommand{\hilmer}[2][]{\todo[inline,color=yellow!70, #1]{\emph{Hilmer:} #2}}
 % Paper metadata (use plain text, for PDF inclusion and later
 % re-using, if desired).  Use \emtpyauthor when submitting for review
@@ -164,6 +171,8 @@
    }
 }
+
+
 \maketitle
 %------------------------------------------------------------------------------
 \input{content/00-abstract}