Commit 9b790fdf935e09d8689ab9a97ab9d3d968e5f071

Authored by Paulo Meireles
2 parents 90070b77 5c521b48

Merge branch 'master' of softwarepublico.gov.br:softwarepublico/articles

ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
@@ -4,6 +4,24 @@ papersize: a4 @@ -4,6 +4,24 @@ papersize: a4
4 geometry: "left=1in,right=1.5in" 4 geometry: "left=1in,right=1.5in"
5 --- 5 ---
6 6
  7 +## Authors
  8 +
  9 +__Diego Camarinha__ is a masters student at IME - The Institute of Mathematics and
  10 +Statistics of the Sao Paulo University. His research interests include software
  11 +engineering, computer networks and source code metrics. He worked for two years
  12 +with the Brazilian federal government as a senior developer. Contact him at
  13 +diegoamc@ime.usp.br.
  14 +
  15 +__Rodrigo Siqueira__ is a masters student at IME - The Institute of Mathematics and
  16 +Statistics of the Sao Paulo University. His research interests include software
  17 +engineering, operating system and computer architecture. He worked for two years
  18 +with the Brazilian federal government as a coach and developer. Contact him at
  19 +siqueira@ime.usp.br.
  20 +
  21 +__Melissa Wen__ is a software developer. She worked on SPB project as a senior developer and also served as professor of Computer Science at UFBA - The Federal University of Bahia. Her areas of interest include software engineering and open source software development. Contact her at melissa.srw@gmail.com .
  22 +
  23 +__Paulo Meirelles__ received his Ph.D. in Computer Science from the Institute of Mathematics and Statistics at the University of São Paulo. He is a full-time Professor at the University of Brasilia, and coordinated the new SPB Portal project. His research interest areas are: Free Software, Agile methods, Static analysis, and Source code metrics. Contact him at paulormm@ime.usp.br.
  24 +
7 ## Abstract 25 ## Abstract
8 26
9 For many software development teams, the first things that come to mind 27 For many software development teams, the first things that come to mind
@@ -28,11 +46,12 @@ government, but also to society as a whole. The government could reduce both @@ -28,11 +46,12 @@ government, but also to society as a whole. The government could reduce both
28 the bureaucracy of using the same software in government agencies and the cost 46 the bureaucracy of using the same software in government agencies and the cost
29 of developing similar software projects. The society gained a mechanism of 47 of developing similar software projects. The society gained a mechanism of
30 transparency and collaboration, since anyone can check the government 48 transparency and collaboration, since anyone can check the government
31 -expenses on software and contribute to project communities. To achieve these  
32 -goals, rather than writing everything from scratch, we decided to integrate  
33 -several free software tools such as Noosfero (www.noosfero.org), Gitlab  
34 -(www.gitlab.com), Mailman (www.gnu.org/software/mailman), Mezuro  
35 -(www.mezuro.org), and Colab (www.github.com/colab). 49 +expenses on software (from information available on the SPB Portal) and
  50 +contribute to project communities. To achieve these goals, rather than writing
  51 +everything from scratch, we decided to integrate several free software tools
  52 +such as Noosfero (www.noosfero.org), Gitlab (www.gitlab.com), Mailman
  53 +(www.gnu.org/software/mailman), Mezuro (www.mezuro.org), and Colab
  54 +(www.github.com/colab).
36 55
37 The project started in a presidential election year and everyone involved was 56 The project started in a presidential election year and everyone involved was
38 under pressure to show results. Even with the re-election of the Brazilian 57 under pressure to show results. Even with the re-election of the Brazilian
@@ -62,10 +81,10 @@ with professors, master students, professional designers and senior developers @@ -62,10 +81,10 @@ with professors, master students, professional designers and senior developers
62 from the Brazilian free software community. Undergraduate students received a 81 from the Brazilian free software community. Undergraduate students received a
63 scholarship and, for most of them, this R&D project was their first 82 scholarship and, for most of them, this R&D project was their first
64 professional experience. Senior developers and master students had two 83 professional experience. Senior developers and master students had two
65 -important contributions to the project: transfer knowledge to undergraduate  
66 -students and more complex tasks. Finally, professors were responsible for  
67 -interacting with the Brazilian government and controlling political pressures  
68 -applied to the project. 84 +important contributions to the project: to transfer knowledge to undergraduate
  85 +students and to deal with more complex tasks. Finally, professors were
  86 +responsible for interacting with the Brazilian government and controlling
  87 +political pressures applied to the project.
69 88
70 Moreover, we needed to communicate with two independent groups of government 89 Moreover, we needed to communicate with two independent groups of government
71 representatives. Requirement analysts were real representatives of the 90 representatives. Requirement analysts were real representatives of the
@@ -83,12 +102,12 @@ the government expectations and to provide quick response to their requests, @@ -83,12 +102,12 @@ the government expectations and to provide quick response to their requests,
83 which were influenced most of the time by the uncertainties of the project's 102 which were influenced most of the time by the uncertainties of the project's
84 continuity. We believed we would keep the project alive, even in a politically 103 continuity. We believed we would keep the project alive, even in a politically
85 unstable and technically complex scenario. For this reason, we focused on 104 unstable and technically complex scenario. For this reason, we focused on
86 -automating the deploy process; for instance, one of our senior  
87 -developers created a Chef-Server (www.chef.io/chef) front-end tool called Chake  
88 -(www.gitlab.com/terceiro/chake) to help us manage the multiple hosts needed for  
89 -the project. We also formed a specific team dedicated to the deployment  
90 -process. This team was responsible for maturing our CD pipeline, giving us  
91 -confidence and agility to comply with government requirements. 105 +automating the deploy process; for instance, one of our senior developers
  106 +created a tool called Chake (www.gitlab.com/terceiro/chake) to help us manage
  107 +multiple hosts without the need for a Chef-Server (www.chef.io/chef). We also
  108 +formed a specific team dedicated to the deployment process. This team was
  109 +responsible for maturing our CD pipeline, giving us confidence and agility to
  110 +comply with government requirements.
92 111
93 In this report, we describe our CD pipeline and how it improved our delivery 112 In this report, we describe our CD pipeline and how it improved our delivery
94 performance. Continuous Delivery made us adaptable for all requested changes 113 performance. Continuous Delivery made us adaptable for all requested changes
@@ -113,18 +132,17 @@ more new code. @@ -113,18 +132,17 @@ more new code.
113 132
114 The SPB portal consists of more than 10 integrated software projects and each 133 The SPB portal consists of more than 10 integrated software projects and each
115 of them, as well as the entire platform, had to be tested. These software 134 of them, as well as the entire platform, had to be tested. These software
116 -components had their own test suite, maintained and expanded by the development  
117 -team. Communication between all components is orchestrated by Colab, a systems  
118 -integration platform for web applications based on a plugins architecture.  
119 -Therefore, specific plugins have been developed for each portal software  
120 -component, such as Gitlab and Mailman. Each plugin had its own test suite  
121 -and this set also worked as integration test.  
122 -  
123 -Both unit and integration tests provided us the performace and security needed  
124 -to guarantee the stability for components and the platform. If any test suite failed,  
125 -by either a test error or coverage reduction below a certain threshold, the  
126 -pipeline was stopped. Only when all tests pass, the pipeline proceeded to the step of  
127 -preparing the release. 135 +components have their own test suite. Communication between all components is
  136 +orchestrated by Colab, a systems integration platform for web applications based
  137 +on a plugins architecture. Therefore, specific plugins were developed for
  138 +each portal software component, such as Gitlab and Mailman. Each plugin has its
  139 +own test suite and this set also worked as integration tests.
  140 +
  141 +Both unit and integration tests provided us the performance and security needed
  142 +to guarantee the stability for components and the platform. If any test suite
  143 +failed, by either a test error or coverage reduction below a certain threshold,
  144 +the pipeline stopped. Only when all tests passed, the pipeline proceeded to the
  145 +step of preparing the release.
128 146
129 ### Preparing a New Release 147 ### Preparing a New Release
130 148
@@ -177,7 +195,7 @@ as part of the next step in the pipeline. @@ -177,7 +195,7 @@ as part of the next step in the pipeline.
177 195
178 After we deployed a new SPB Portal version in the VE, the government 196 After we deployed a new SPB Portal version in the VE, the government
179 agents were responsible for checking features and bug fixes required by them. If 197 agents were responsible for checking features and bug fixes required by them. If
180 -the requirement analysts identified a problem, they would notified the developers via 198 +the requirement analysts identified a problem, they would notify the developers via
181 comments on the SPB Portal's issue tracker. The problems were fixed and the 199 comments on the SPB Portal's issue tracker. The problems were fixed and the
182 pipeline restarted from scratch. If everything was validated, we moved forward. 200 pipeline restarted from scratch. If everything was validated, we moved forward.
183 201
@@ -206,7 +224,7 @@ leader resulted in new requirements, most of them motivated by political @@ -206,7 +224,7 @@ leader resulted in new requirements, most of them motivated by political
206 needs. These constant changes in requirements and priorities caused discomfort 224 needs. These constant changes in requirements and priorities caused discomfort
207 between the government and the development team. For 225 between the government and the development team. For
208 example, the government leader required a complete layout change because he suddenly decided to make a marketing campaign about the new 226 example, the government leader required a complete layout change because he suddenly decided to make a marketing campaign about the new
209 -SPB portal. Thus the government would use undelivered requirements as a means to justify the 227 +SPB portal. In future meetings, the government would use undelivered requirements as a means to justify the
210 lack of financial support, which was already approved in the first place. We believed that if we took too 228 lack of financial support, which was already approved in the first place. We believed that if we took too
211 long to attend their demands, the project would end. CD helped us keep the 229 long to attend their demands, the project would end. CD helped us keep the
212 production environment up-to-date, even with partial versions of a feature. That 230 production environment up-to-date, even with partial versions of a feature. That
@@ -302,6 +320,10 @@ to part of the infrastructure. More research is required on development protocol @@ -302,6 +320,10 @@ to part of the infrastructure. More research is required on development protocol
302 policies to improve the relation between industry and government, specially 320 policies to improve the relation between industry and government, specially
303 regarding CD. 321 regarding CD.
304 322
  323 +## Acknowledgements
  324 +
  325 +We thank our colleagues, Lucas Kanashiro and Rafael Manzo, and this article's reviewers.
  326 +
305 ## References 327 ## References
306 328
307 1. G. Booch, A. W. Brown, "Collaborative Development Environments", in Advances in Computers, vol. 59, 2003, pp. 1–27. 329 1. G. Booch, A. W. Brown, "Collaborative Development Environments", in Advances in Computers, vol. 59, 2003, pp. 1–27.