Commit 9b790fdf935e09d8689ab9a97ab9d3d968e5f071
Exists in
master
and in
3 other branches
Merge branch 'master' of softwarepublico.gov.br:softwarepublico/articles
Showing
1 changed file
with
51 additions
and
29 deletions
Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
@@ -4,6 +4,24 @@ papersize: a4 | @@ -4,6 +4,24 @@ papersize: a4 | ||
4 | geometry: "left=1in,right=1.5in" | 4 | geometry: "left=1in,right=1.5in" |
5 | --- | 5 | --- |
6 | 6 | ||
7 | +## Authors | ||
8 | + | ||
9 | +__Diego Camarinha__ is a masters student at IME - The Institute of Mathematics and | ||
10 | +Statistics of the Sao Paulo University. His research interests include software | ||
11 | +engineering, computer networks and source code metrics. He worked for two years | ||
12 | +with the Brazilian federal government as a senior developer. Contact him at | ||
13 | +diegoamc@ime.usp.br. | ||
14 | + | ||
15 | +__Rodrigo Siqueira__ is a masters student at IME - The Institute of Mathematics and | ||
16 | +Statistics of the Sao Paulo University. His research interests include software | ||
17 | +engineering, operating system and computer architecture. He worked for two years | ||
18 | +with the Brazilian federal government as a coach and developer. Contact him at | ||
19 | +siqueira@ime.usp.br. | ||
20 | + | ||
21 | +__Melissa Wen__ is a software developer. She worked on SPB project as a senior developer and also served as professor of Computer Science at UFBA - The Federal University of Bahia. Her areas of interest include software engineering and open source software development. Contact her at melissa.srw@gmail.com . | ||
22 | + | ||
23 | +__Paulo Meirelles__ received his Ph.D. in Computer Science from the Institute of Mathematics and Statistics at the University of São Paulo. He is a full-time Professor at the University of Brasilia, and coordinated the new SPB Portal project. His research interest areas are: Free Software, Agile methods, Static analysis, and Source code metrics. Contact him at paulormm@ime.usp.br. | ||
24 | + | ||
7 | ## Abstract | 25 | ## Abstract |
8 | 26 | ||
9 | For many software development teams, the first things that come to mind | 27 | For many software development teams, the first things that come to mind |
@@ -28,11 +46,12 @@ government, but also to society as a whole. The government could reduce both | @@ -28,11 +46,12 @@ government, but also to society as a whole. The government could reduce both | ||
28 | the bureaucracy of using the same software in government agencies and the cost | 46 | the bureaucracy of using the same software in government agencies and the cost |
29 | of developing similar software projects. The society gained a mechanism of | 47 | of developing similar software projects. The society gained a mechanism of |
30 | transparency and collaboration, since anyone can check the government | 48 | transparency and collaboration, since anyone can check the government |
31 | -expenses on software and contribute to project communities. To achieve these | ||
32 | -goals, rather than writing everything from scratch, we decided to integrate | ||
33 | -several free software tools such as Noosfero (www.noosfero.org), Gitlab | ||
34 | -(www.gitlab.com), Mailman (www.gnu.org/software/mailman), Mezuro | ||
35 | -(www.mezuro.org), and Colab (www.github.com/colab). | 49 | +expenses on software (from information available on the SPB Portal) and |
50 | +contribute to project communities. To achieve these goals, rather than writing | ||
51 | +everything from scratch, we decided to integrate several free software tools | ||
52 | +such as Noosfero (www.noosfero.org), Gitlab (www.gitlab.com), Mailman | ||
53 | +(www.gnu.org/software/mailman), Mezuro (www.mezuro.org), and Colab | ||
54 | +(www.github.com/colab). | ||
36 | 55 | ||
37 | The project started in a presidential election year and everyone involved was | 56 | The project started in a presidential election year and everyone involved was |
38 | under pressure to show results. Even with the re-election of the Brazilian | 57 | under pressure to show results. Even with the re-election of the Brazilian |
@@ -62,10 +81,10 @@ with professors, master students, professional designers and senior developers | @@ -62,10 +81,10 @@ with professors, master students, professional designers and senior developers | ||
62 | from the Brazilian free software community. Undergraduate students received a | 81 | from the Brazilian free software community. Undergraduate students received a |
63 | scholarship and, for most of them, this R&D project was their first | 82 | scholarship and, for most of them, this R&D project was their first |
64 | professional experience. Senior developers and master students had two | 83 | professional experience. Senior developers and master students had two |
65 | -important contributions to the project: transfer knowledge to undergraduate | ||
66 | -students and more complex tasks. Finally, professors were responsible for | ||
67 | -interacting with the Brazilian government and controlling political pressures | ||
68 | -applied to the project. | 84 | +important contributions to the project: to transfer knowledge to undergraduate |
85 | +students and to deal with more complex tasks. Finally, professors were | ||
86 | +responsible for interacting with the Brazilian government and controlling | ||
87 | +political pressures applied to the project. | ||
69 | 88 | ||
70 | Moreover, we needed to communicate with two independent groups of government | 89 | Moreover, we needed to communicate with two independent groups of government |
71 | representatives. Requirement analysts were real representatives of the | 90 | representatives. Requirement analysts were real representatives of the |
@@ -83,12 +102,12 @@ the government expectations and to provide quick response to their requests, | @@ -83,12 +102,12 @@ the government expectations and to provide quick response to their requests, | ||
83 | which were influenced most of the time by the uncertainties of the project's | 102 | which were influenced most of the time by the uncertainties of the project's |
84 | continuity. We believed we would keep the project alive, even in a politically | 103 | continuity. We believed we would keep the project alive, even in a politically |
85 | unstable and technically complex scenario. For this reason, we focused on | 104 | unstable and technically complex scenario. For this reason, we focused on |
86 | -automating the deploy process; for instance, one of our senior | ||
87 | -developers created a Chef-Server (www.chef.io/chef) front-end tool called Chake | ||
88 | -(www.gitlab.com/terceiro/chake) to help us manage the multiple hosts needed for | ||
89 | -the project. We also formed a specific team dedicated to the deployment | ||
90 | -process. This team was responsible for maturing our CD pipeline, giving us | ||
91 | -confidence and agility to comply with government requirements. | 105 | +automating the deploy process; for instance, one of our senior developers |
106 | +created a tool called Chake (www.gitlab.com/terceiro/chake) to help us manage | ||
107 | +multiple hosts without the need for a Chef-Server (www.chef.io/chef). We also | ||
108 | +formed a specific team dedicated to the deployment process. This team was | ||
109 | +responsible for maturing our CD pipeline, giving us confidence and agility to | ||
110 | +comply with government requirements. | ||
92 | 111 | ||
93 | In this report, we describe our CD pipeline and how it improved our delivery | 112 | In this report, we describe our CD pipeline and how it improved our delivery |
94 | performance. Continuous Delivery made us adaptable for all requested changes | 113 | performance. Continuous Delivery made us adaptable for all requested changes |
@@ -113,18 +132,17 @@ more new code. | @@ -113,18 +132,17 @@ more new code. | ||
113 | 132 | ||
114 | The SPB portal consists of more than 10 integrated software projects and each | 133 | The SPB portal consists of more than 10 integrated software projects and each |
115 | of them, as well as the entire platform, had to be tested. These software | 134 | of them, as well as the entire platform, had to be tested. These software |
116 | -components had their own test suite, maintained and expanded by the development | ||
117 | -team. Communication between all components is orchestrated by Colab, a systems | ||
118 | -integration platform for web applications based on a plugins architecture. | ||
119 | -Therefore, specific plugins have been developed for each portal software | ||
120 | -component, such as Gitlab and Mailman. Each plugin had its own test suite | ||
121 | -and this set also worked as integration test. | ||
122 | - | ||
123 | -Both unit and integration tests provided us the performace and security needed | ||
124 | -to guarantee the stability for components and the platform. If any test suite failed, | ||
125 | -by either a test error or coverage reduction below a certain threshold, the | ||
126 | -pipeline was stopped. Only when all tests pass, the pipeline proceeded to the step of | ||
127 | -preparing the release. | 135 | +components have their own test suite. Communication between all components is |
136 | +orchestrated by Colab, a systems integration platform for web applications based | ||
137 | +on a plugins architecture. Therefore, specific plugins were developed for | ||
138 | +each portal software component, such as Gitlab and Mailman. Each plugin has its | ||
139 | +own test suite and this set also worked as integration tests. | ||
140 | + | ||
141 | +Both unit and integration tests provided us the performance and security needed | ||
142 | +to guarantee the stability for components and the platform. If any test suite | ||
143 | +failed, by either a test error or coverage reduction below a certain threshold, | ||
144 | +the pipeline stopped. Only when all tests passed, the pipeline proceeded to the | ||
145 | +step of preparing the release. | ||
128 | 146 | ||
129 | ### Preparing a New Release | 147 | ### Preparing a New Release |
130 | 148 | ||
@@ -177,7 +195,7 @@ as part of the next step in the pipeline. | @@ -177,7 +195,7 @@ as part of the next step in the pipeline. | ||
177 | 195 | ||
178 | After we deployed a new SPB Portal version in the VE, the government | 196 | After we deployed a new SPB Portal version in the VE, the government |
179 | agents were responsible for checking features and bug fixes required by them. If | 197 | agents were responsible for checking features and bug fixes required by them. If |
180 | -the requirement analysts identified a problem, they would notified the developers via | 198 | +the requirement analysts identified a problem, they would notify the developers via |
181 | comments on the SPB Portal's issue tracker. The problems were fixed and the | 199 | comments on the SPB Portal's issue tracker. The problems were fixed and the |
182 | pipeline restarted from scratch. If everything was validated, we moved forward. | 200 | pipeline restarted from scratch. If everything was validated, we moved forward. |
183 | 201 | ||
@@ -206,7 +224,7 @@ leader resulted in new requirements, most of them motivated by political | @@ -206,7 +224,7 @@ leader resulted in new requirements, most of them motivated by political | ||
206 | needs. These constant changes in requirements and priorities caused discomfort | 224 | needs. These constant changes in requirements and priorities caused discomfort |
207 | between the government and the development team. For | 225 | between the government and the development team. For |
208 | example, the government leader required a complete layout change because he suddenly decided to make a marketing campaign about the new | 226 | example, the government leader required a complete layout change because he suddenly decided to make a marketing campaign about the new |
209 | -SPB portal. Thus the government would use undelivered requirements as a means to justify the | 227 | +SPB portal. In future meetings, the government would use undelivered requirements as a means to justify the |
210 | lack of financial support, which was already approved in the first place. We believed that if we took too | 228 | lack of financial support, which was already approved in the first place. We believed that if we took too |
211 | long to attend their demands, the project would end. CD helped us keep the | 229 | long to attend their demands, the project would end. CD helped us keep the |
212 | production environment up-to-date, even with partial versions of a feature. That | 230 | production environment up-to-date, even with partial versions of a feature. That |
@@ -302,6 +320,10 @@ to part of the infrastructure. More research is required on development protocol | @@ -302,6 +320,10 @@ to part of the infrastructure. More research is required on development protocol | ||
302 | policies to improve the relation between industry and government, specially | 320 | policies to improve the relation between industry and government, specially |
303 | regarding CD. | 321 | regarding CD. |
304 | 322 | ||
323 | +## Acknowledgements | ||
324 | + | ||
325 | +We thank our colleagues, Lucas Kanashiro and Rafael Manzo, and this article's reviewers. | ||
326 | + | ||
305 | ## References | 327 | ## References |
306 | 328 | ||
307 | 1. G. Booch, A. W. Brown, "Collaborative Development Environments", in Advances in Computers, vol. 59, 2003, pp. 1–27. | 329 | 1. G. Booch, A. W. Brown, "Collaborative Development Environments", in Advances in Computers, vol. 59, 2003, pp. 1–27. |