Last review

Paulo Meireles
1 parent 5317e8b5
Showing 1 changed file with 71 additions and 69 deletions Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
@@ -12,28 +12,31 @@ geometry: &quot;left=1in,right=1.5in&quot;
 __Rodrigo Siqueira__ is a Computer Science MSc student at the Institute of
 Mathematics and Statistics of the University of São Paulo (IME-USP). His
 research interests include software engineering, operating system, and computer
-architecture. He worked for two years with the Brazilian federal government as
-a coach and developer.  Contact him at siqueira@ime.usp.br.
-
-__Diego Camarinha__ is a CS MSc masters student at IME-USP. His research
-interests include software engineering, computer networks, and source code
-metrics. He worked for two years with the Brazilian federal government as a
-senior developer. Contact him at diegoamc@ime.usp.br.
-
-__Melissa Wen__ is a software developer. She worked on the SPB project as a
-senior developer and also served as a Computer Science lecturer at the Federal
-University of Bahia. Her areas of interest include software engineering and
-open source software development. Contact her at melissa.srw@gmail.com.
-
-__Paulo Meirelles__ received his Ph.D. in Computer Science from IME-USP. He is
-a full-time Professor at the University of Brasilia and coordinated the new SPB
-portal project. His research interests are: Free Software, Agile methods,
-Static analysis, and Source code metrics. Contact him at paulormm@ime.usp.br.
-
-__Fabio Kon__ is a Full Professor of Computer Science at IME-USP and
-Editor-in-Chief of the SpringerOpen Journal of Internet Services and
-Applications. His research interests include Agile Software Development, Smart
-Cities, and Distributed Systems. Contact him at fabio.kon@ime.usp.br.
+architecture. He worked for two years with the SPB project as a coach and
+developer. Contact him at siqueira@ime.usp.br.
+
+__Diego Camarinha__ is a Computer Science MSc masters student at IME-USP. His
+research interests include software engineering, computer networks, and source
+code metrics. He worked for two years with the SPB project as a senior
+developer. Contact him at diegoamc@ime.usp.br.
+
+__Melissa Wen__ is a Computer Science MSc masters student at IME-USP. She
+worked on the SPB project as a senior developer and was core developer of
+Noosfero social networking platform at Colivre. Her areas of interest include
+software engineering and open source software development. Contact her at
+melissa@colivre.coop.br.
+
+__Paulo Meirelles__ is a full-time Professor of Software Engineering at the
+University of Brasilia and researcher at the Free/Libre/Open Source Software
+Competence Center (CCSL) at IME-USP. He was the head of the new SPB portal
+development. His research interests include Free Software development, Agile
+Software Development, and Source code metrics. Contact him at paulormm@unb.br.
+
+__Fabio Kon__ is a Full Professor of Computer Science at IME-USP, co-founder of
+the CCSL at IME-USP, and Editor-in-Chief of the SpringerOpen Journal of
+Internet Services and Applications. His research interests include Agile
+Software Development, Smart Cities, and Distributed Systems. Contact him at
+fabio.kon@ime.usp.br.
 ## Abstract
@@ -51,8 +54,8 @@ valuable for readers in a variety of situations.
 From 2014 to 2016, we worked on a thirty-month-long Brazilian government
 project to modernize the Brazilian Public Software (SPB) portal
 (www.softwarepublico.gov.br) [1]. This project was a partnership between the
-_Ministry of Planning, Budget, and Management_ and two public universities:
-University of Brasília and University of São Paulo.
+_Ministry of Planning, Budget, and Management (MPOG)_ and two public
+universities: University of Brasília and University of São Paulo.
 During this time, the SPB portal evolved into a Collaborative Development
 Environment (CDE), which brought significant benefits for the government and
@@ -74,9 +77,9 @@ trust relationships is yet another of its benefits [2,3].
 SPB is a governmental program created to foster sharing and collaboration on
 Open Source Software (OSS) development for the Brazilian public administration.
-The ministry managed both software requirements and server infrastructure.
-However, its hierarchical and traditional processes made them unfamiliar with
-new software development techniques, such as CD. All our requests had to pass
+The MPOG managed both software requirements and server infrastructure. However,
+its hierarchical and traditional processes made them unfamiliar with new
+software development techniques, such as CD. All our requests had to pass
 through layers of bureaucracy before being answered; accessing their
 infrastructure to deploy software was not different.  The difficulties were
 aggravated because the new SPB portal is an unprecedented platform in the
@@ -98,16 +101,16 @@ the government deployment process.
 First, our development team was not the typical one, consisting mainly of
 undergraduate interns supported by senior developers and designers, all
 coordinated by two professors. Our unconventional team structure and
-organization was considered as "unprofessional" by ministry standards with
+organization was considered as "unprofessional" by government standards with
 regard to  time and resource allocation, accountability, and team continuity.
 On the government side, the SPB portal evolution was the first software
-development collaboration involving universities and the ministry staff,
-raising disbelief.
+development collaboration involving universities and the MPOG staff, raising
+disbelief.
 Second, our team approached software deployment differently from the
 government. We believed frequent delivery is better for the project’s success.
-In contrast, the ministry is used to the idea of a single deployment at the end
-of the project, and neither their bureaucratic structure nor their technical
+In contrast, the MPOG is used to the idea of a single deployment at the end of
+the project, and neither their bureaucratic structure nor their technical
 expertise was conducive to this style of work. That was hampering the benefits
 of the tool and preventing us from showing off the fruits of the project to
 those responsible for evaluating it.
@@ -135,9 +138,9 @@ We built a system-of-systems [5] adapting and integrating five existing OSS
 projects: Colab (www.github.com/colab), Noosfero (www.noosfero.org), Gitlab
 (www.gitlab.com), Mezuro (www.mezuro.org), and Mailman (www.list.org). Colab
 orchestrates these multiple systems using a plugin architecture and allowed us
-to smoothly provide a unified interface to final users, including SSO and
-global searches [1]. All these integrated systems involve a total of 106,253
-commits and 1,347,421 lines of code.
+to smoothly provide a unified interface to final users, including single
+sign-on and global searches [1]. All these integrated systems involve a total
+of 106,253 commits and 1,347,421 lines of code.
 ![The SPB Deployment Pipeline.](figures/pipeline.png)
@@ -179,8 +182,8 @@ available for our deployment scripts.
 ### Validation Environment 
 The Validation Environment (VE) is a replica of the Production Environment (PE)
-with anonymised data, and access restricted to ministry staff and our DevOps
-team. To configure this environment, we used Chef (www.chef.io) and Chake, a
+with anonymised data, and access restricted to MPOG staff and our DevOps team.
+To configure this environment, we used Chef (www.chef.io) and Chake, a
 serverless configuration tool created by our team
 (www.github.com/terceiro/chake) to maintain environment consistency,
 simplifying the deployment process.
@@ -188,7 +191,7 @@ simplifying the deployment process.
 ### Acceptance Tests
 After a new SPB portal VE deployment, we used the environment to verify the
-integrity of the entire portal. Government staff also checked the new features,
+integrity of the entire portal. MPOG staff also checked the new features,
 required changes, and bug fixes. If they identified a problem, they would
 notify developers via comments on the SPB portal issue tracker, prompting the
 team to fix it and restart the pipeline. Otherwise, we could move forward.
@@ -215,13 +218,13 @@ benefits.
 ### Strengthening Trust in the Relationship with the Government
-CD helped strengthen trust between developers and the government staff. Before
-using CD, they could validate features developed only at the end of the release
+CD helped strengthen trust between developers and the MPOG staff. Before using
+CD, they could validate features developed only at the end of the release
 cycle, usually every four months. With the implementation of CD, intermediate
 and candidate versions became available, allowing them to perform small
 validations over time. Constant monitoring of the development work brought
-greater assurance to the ministry leaders and improved the interactions with
-our team.
+greater assurance to the MPOG leaders and improved the interactions with our
+team.
 ### Responsiveness to Change
@@ -238,13 +241,13 @@ confident that the project would last a little longer.
 ### Shared Responsibility
-According to the conventional ministry process, the development team could not
+According to the conventional MPOG process, the development team could not
 track what happened to the code after its delivery, since their employees were
 the only ones responsible for deployment. The implementation of CD made our
 development team feel equally responsible for what was getting into production
 and take ownership of the project [4].
-Interestingly, the CD pipeline had the same effect on the ministry staff. They
+Interestingly, the CD pipeline had the same effect on the MPOG staff. They
 became more engaged in the whole process, opening and discussing issues during
 the evolution of the platform. Additionally, developers worked to improve the
 CD pipeline and speed up the process of making new features available in the
@@ -253,19 +256,18 @@ production environment.
 ### Synchronization Between Government and Development
 The CD pipeline performance depended on the synchronization between our
-development team and the ministry staff: each party had to be prepared to take
-action as soon as the other concluded a given task. Initially, the ministry
-staff did not contemplate this in their schedule which, combined with the
-bureaucracy in providing access to the PE (up to 3 days), resulted in
-significant delays in the validation of new features. The use of an explicit CD
-pipeline helped us to identify critical points of delay, and increase our
-productivity.
+development team and the MPOG staff: each party had to be prepared to take
+action as soon as the other concluded a given task. Initially, the MPOG staff
+did not contemplate this in their schedule which, combined with the bureaucracy
+in providing access to the PE (up to 3 days), resulted in significant delays in
+the validation of new features. The use of an explicit CD pipeline helped us to
+identify critical points of delay, and increase our productivity.
 ## Lessons Learned
 Due to the successful building of the CD pipeline, we not only overcame the
-challenges we described above but also improved the ministry deployment process
-and kept the project alive until its successful conclusion. We now look at the
+challenges we described above but also improved the MPOG deployment process and
+kept the project alive until its successful conclusion. We now look at the
 lessons we learned, which can be leveraged by readers in other situations.
 ### Build CD From Scratch
@@ -288,38 +290,38 @@ process on-the-fly.
 ### Overcoming Mistrust
-Adopting an unfamiliar approach requires trust. ministry staff, traditionally,
+Adopting an unfamiliar approach requires trust. MPOG staff, traditionally,
 expected and were prepared to validate and deploy a single deliverable.
 However, the steady growth of SPB complexity made large deliveries
 unsustainable. The CD approach was necessary [4]. Therefore, we developed the
 following line of action to enable this new dynamics:
 1. _Demonstrate actual results, instead of simply reporting them._ Initially,
-we did not have access to the ministry infrastructure, so we created our own
+we did not have access to the government infrastructure, so we created our own
 validation environment. Thus, we were able to follow the CD pipeline until
 production deployment, when we faced two problems. First, our pace of
 intermediate deliveries was faster than the deployment to production by the
-government staff. Second, specific issues of the governmental infrastructure
-made some validated features not work as expected in the PE. That situation
-gave us arguments to negotiate access to the PE.
+MPOG staff. Second, specific issues of the governmental infrastructure made
+some validated features not work as expected in the PE. That situation gave us
+arguments to negotiate access to the PE.
 2. _Make project management transparent and collaborative for government
-staff._ Allowing the ministry staff to track our development process showed
-them we were fulfilling our commitments. They started to interact more actively
-in the generation of versions and became involved in the CD. After
-understanding it, the ministry staff helped us negotiate access to a VE with
-the ministry leaders, creating an isolated replica of the PE.
+staff._ Allowing the MPOG staff to track our development process showed them we
+were fulfilling our commitments. They started to interact more actively in the
+generation of versions and became involved in the CD. After understanding it,
+the MPOG staff helped us negotiate access to a VE with the MPOG leaders,
+creating an isolated replica of the PE.
 3. _Gain the confidence of government staff._ With the PE replica, we were able
-to run the entire pipeline and win the trust of the ministry staff involved in
-the process. They saw the mobilization and responsiveness of our team to
-generate each new version package. They also recognized the quality of our work
-and deployment process. In the end, the ministry staff realized that it would
-be beneficial for the project if they granted us access to the infrastructure,
+to run the entire pipeline and win the trust of the MPOG staff involved in the
+process. They saw the mobilization and responsiveness of our team to generate
+each new version package. They also recognized the quality of our work and
+deployment process. In the end, the MPOG staff realized that it would be
+beneficial for the project if they granted us access to the infrastructure,
 both VE and PE.
 In summary, we encourage the use of a well-thought CD pipeline as a means to
-gain trust in software development projects with large organizations.
+gain trust in software development projects with government organizations.
 ## References