Commit 9b790fdf935e09d8689ab9a97ab9d3d968e5f071
Exists in
master
and in
3 other branches
Merge branch 'master' of softwarepublico.gov.br:softwarepublico/articles
Showing
1 changed file
with
51 additions
and
29 deletions
Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
| @@ -4,6 +4,24 @@ papersize: a4 | @@ -4,6 +4,24 @@ papersize: a4 | ||
| 4 | geometry: "left=1in,right=1.5in" | 4 | geometry: "left=1in,right=1.5in" |
| 5 | --- | 5 | --- |
| 6 | 6 | ||
| 7 | +## Authors | ||
| 8 | + | ||
| 9 | +__Diego Camarinha__ is a masters student at IME - The Institute of Mathematics and | ||
| 10 | +Statistics of the Sao Paulo University. His research interests include software | ||
| 11 | +engineering, computer networks and source code metrics. He worked for two years | ||
| 12 | +with the Brazilian federal government as a senior developer. Contact him at | ||
| 13 | +diegoamc@ime.usp.br. | ||
| 14 | + | ||
| 15 | +__Rodrigo Siqueira__ is a masters student at IME - The Institute of Mathematics and | ||
| 16 | +Statistics of the Sao Paulo University. His research interests include software | ||
| 17 | +engineering, operating system and computer architecture. He worked for two years | ||
| 18 | +with the Brazilian federal government as a coach and developer. Contact him at | ||
| 19 | +siqueira@ime.usp.br. | ||
| 20 | + | ||
| 21 | +__Melissa Wen__ is a software developer. She worked on SPB project as a senior developer and also served as professor of Computer Science at UFBA - The Federal University of Bahia. Her areas of interest include software engineering and open source software development. Contact her at melissa.srw@gmail.com . | ||
| 22 | + | ||
| 23 | +__Paulo Meirelles__ received his Ph.D. in Computer Science from the Institute of Mathematics and Statistics at the University of São Paulo. He is a full-time Professor at the University of Brasilia, and coordinated the new SPB Portal project. His research interest areas are: Free Software, Agile methods, Static analysis, and Source code metrics. Contact him at paulormm@ime.usp.br. | ||
| 24 | + | ||
| 7 | ## Abstract | 25 | ## Abstract |
| 8 | 26 | ||
| 9 | For many software development teams, the first things that come to mind | 27 | For many software development teams, the first things that come to mind |
| @@ -28,11 +46,12 @@ government, but also to society as a whole. The government could reduce both | @@ -28,11 +46,12 @@ government, but also to society as a whole. The government could reduce both | ||
| 28 | the bureaucracy of using the same software in government agencies and the cost | 46 | the bureaucracy of using the same software in government agencies and the cost |
| 29 | of developing similar software projects. The society gained a mechanism of | 47 | of developing similar software projects. The society gained a mechanism of |
| 30 | transparency and collaboration, since anyone can check the government | 48 | transparency and collaboration, since anyone can check the government |
| 31 | -expenses on software and contribute to project communities. To achieve these | ||
| 32 | -goals, rather than writing everything from scratch, we decided to integrate | ||
| 33 | -several free software tools such as Noosfero (www.noosfero.org), Gitlab | ||
| 34 | -(www.gitlab.com), Mailman (www.gnu.org/software/mailman), Mezuro | ||
| 35 | -(www.mezuro.org), and Colab (www.github.com/colab). | 49 | +expenses on software (from information available on the SPB Portal) and |
| 50 | +contribute to project communities. To achieve these goals, rather than writing | ||
| 51 | +everything from scratch, we decided to integrate several free software tools | ||
| 52 | +such as Noosfero (www.noosfero.org), Gitlab (www.gitlab.com), Mailman | ||
| 53 | +(www.gnu.org/software/mailman), Mezuro (www.mezuro.org), and Colab | ||
| 54 | +(www.github.com/colab). | ||
| 36 | 55 | ||
| 37 | The project started in a presidential election year and everyone involved was | 56 | The project started in a presidential election year and everyone involved was |
| 38 | under pressure to show results. Even with the re-election of the Brazilian | 57 | under pressure to show results. Even with the re-election of the Brazilian |
| @@ -62,10 +81,10 @@ with professors, master students, professional designers and senior developers | @@ -62,10 +81,10 @@ with professors, master students, professional designers and senior developers | ||
| 62 | from the Brazilian free software community. Undergraduate students received a | 81 | from the Brazilian free software community. Undergraduate students received a |
| 63 | scholarship and, for most of them, this R&D project was their first | 82 | scholarship and, for most of them, this R&D project was their first |
| 64 | professional experience. Senior developers and master students had two | 83 | professional experience. Senior developers and master students had two |
| 65 | -important contributions to the project: transfer knowledge to undergraduate | ||
| 66 | -students and more complex tasks. Finally, professors were responsible for | ||
| 67 | -interacting with the Brazilian government and controlling political pressures | ||
| 68 | -applied to the project. | 84 | +important contributions to the project: to transfer knowledge to undergraduate |
| 85 | +students and to deal with more complex tasks. Finally, professors were | ||
| 86 | +responsible for interacting with the Brazilian government and controlling | ||
| 87 | +political pressures applied to the project. | ||
| 69 | 88 | ||
| 70 | Moreover, we needed to communicate with two independent groups of government | 89 | Moreover, we needed to communicate with two independent groups of government |
| 71 | representatives. Requirement analysts were real representatives of the | 90 | representatives. Requirement analysts were real representatives of the |
| @@ -83,12 +102,12 @@ the government expectations and to provide quick response to their requests, | @@ -83,12 +102,12 @@ the government expectations and to provide quick response to their requests, | ||
| 83 | which were influenced most of the time by the uncertainties of the project's | 102 | which were influenced most of the time by the uncertainties of the project's |
| 84 | continuity. We believed we would keep the project alive, even in a politically | 103 | continuity. We believed we would keep the project alive, even in a politically |
| 85 | unstable and technically complex scenario. For this reason, we focused on | 104 | unstable and technically complex scenario. For this reason, we focused on |
| 86 | -automating the deploy process; for instance, one of our senior | ||
| 87 | -developers created a Chef-Server (www.chef.io/chef) front-end tool called Chake | ||
| 88 | -(www.gitlab.com/terceiro/chake) to help us manage the multiple hosts needed for | ||
| 89 | -the project. We also formed a specific team dedicated to the deployment | ||
| 90 | -process. This team was responsible for maturing our CD pipeline, giving us | ||
| 91 | -confidence and agility to comply with government requirements. | 105 | +automating the deploy process; for instance, one of our senior developers |
| 106 | +created a tool called Chake (www.gitlab.com/terceiro/chake) to help us manage | ||
| 107 | +multiple hosts without the need for a Chef-Server (www.chef.io/chef). We also | ||
| 108 | +formed a specific team dedicated to the deployment process. This team was | ||
| 109 | +responsible for maturing our CD pipeline, giving us confidence and agility to | ||
| 110 | +comply with government requirements. | ||
| 92 | 111 | ||
| 93 | In this report, we describe our CD pipeline and how it improved our delivery | 112 | In this report, we describe our CD pipeline and how it improved our delivery |
| 94 | performance. Continuous Delivery made us adaptable for all requested changes | 113 | performance. Continuous Delivery made us adaptable for all requested changes |
| @@ -113,18 +132,17 @@ more new code. | @@ -113,18 +132,17 @@ more new code. | ||
| 113 | 132 | ||
| 114 | The SPB portal consists of more than 10 integrated software projects and each | 133 | The SPB portal consists of more than 10 integrated software projects and each |
| 115 | of them, as well as the entire platform, had to be tested. These software | 134 | of them, as well as the entire platform, had to be tested. These software |
| 116 | -components had their own test suite, maintained and expanded by the development | ||
| 117 | -team. Communication between all components is orchestrated by Colab, a systems | ||
| 118 | -integration platform for web applications based on a plugins architecture. | ||
| 119 | -Therefore, specific plugins have been developed for each portal software | ||
| 120 | -component, such as Gitlab and Mailman. Each plugin had its own test suite | ||
| 121 | -and this set also worked as integration test. | ||
| 122 | - | ||
| 123 | -Both unit and integration tests provided us the performace and security needed | ||
| 124 | -to guarantee the stability for components and the platform. If any test suite failed, | ||
| 125 | -by either a test error or coverage reduction below a certain threshold, the | ||
| 126 | -pipeline was stopped. Only when all tests pass, the pipeline proceeded to the step of | ||
| 127 | -preparing the release. | 135 | +components have their own test suite. Communication between all components is |
| 136 | +orchestrated by Colab, a systems integration platform for web applications based | ||
| 137 | +on a plugins architecture. Therefore, specific plugins were developed for | ||
| 138 | +each portal software component, such as Gitlab and Mailman. Each plugin has its | ||
| 139 | +own test suite and this set also worked as integration tests. | ||
| 140 | + | ||
| 141 | +Both unit and integration tests provided us the performance and security needed | ||
| 142 | +to guarantee the stability for components and the platform. If any test suite | ||
| 143 | +failed, by either a test error or coverage reduction below a certain threshold, | ||
| 144 | +the pipeline stopped. Only when all tests passed, the pipeline proceeded to the | ||
| 145 | +step of preparing the release. | ||
| 128 | 146 | ||
| 129 | ### Preparing a New Release | 147 | ### Preparing a New Release |
| 130 | 148 | ||
| @@ -177,7 +195,7 @@ as part of the next step in the pipeline. | @@ -177,7 +195,7 @@ as part of the next step in the pipeline. | ||
| 177 | 195 | ||
| 178 | After we deployed a new SPB Portal version in the VE, the government | 196 | After we deployed a new SPB Portal version in the VE, the government |
| 179 | agents were responsible for checking features and bug fixes required by them. If | 197 | agents were responsible for checking features and bug fixes required by them. If |
| 180 | -the requirement analysts identified a problem, they would notified the developers via | 198 | +the requirement analysts identified a problem, they would notify the developers via |
| 181 | comments on the SPB Portal's issue tracker. The problems were fixed and the | 199 | comments on the SPB Portal's issue tracker. The problems were fixed and the |
| 182 | pipeline restarted from scratch. If everything was validated, we moved forward. | 200 | pipeline restarted from scratch. If everything was validated, we moved forward. |
| 183 | 201 | ||
| @@ -206,7 +224,7 @@ leader resulted in new requirements, most of them motivated by political | @@ -206,7 +224,7 @@ leader resulted in new requirements, most of them motivated by political | ||
| 206 | needs. These constant changes in requirements and priorities caused discomfort | 224 | needs. These constant changes in requirements and priorities caused discomfort |
| 207 | between the government and the development team. For | 225 | between the government and the development team. For |
| 208 | example, the government leader required a complete layout change because he suddenly decided to make a marketing campaign about the new | 226 | example, the government leader required a complete layout change because he suddenly decided to make a marketing campaign about the new |
| 209 | -SPB portal. Thus the government would use undelivered requirements as a means to justify the | 227 | +SPB portal. In future meetings, the government would use undelivered requirements as a means to justify the |
| 210 | lack of financial support, which was already approved in the first place. We believed that if we took too | 228 | lack of financial support, which was already approved in the first place. We believed that if we took too |
| 211 | long to attend their demands, the project would end. CD helped us keep the | 229 | long to attend their demands, the project would end. CD helped us keep the |
| 212 | production environment up-to-date, even with partial versions of a feature. That | 230 | production environment up-to-date, even with partial versions of a feature. That |
| @@ -302,6 +320,10 @@ to part of the infrastructure. More research is required on development protocol | @@ -302,6 +320,10 @@ to part of the infrastructure. More research is required on development protocol | ||
| 302 | policies to improve the relation between industry and government, specially | 320 | policies to improve the relation between industry and government, specially |
| 303 | regarding CD. | 321 | regarding CD. |
| 304 | 322 | ||
| 323 | +## Acknowledgements | ||
| 324 | + | ||
| 325 | +We thank our colleagues, Lucas Kanashiro and Rafael Manzo, and this article's reviewers. | ||
| 326 | + | ||
| 305 | ## References | 327 | ## References |
| 306 | 328 | ||
| 307 | 1. G. Booch, A. W. Brown, "Collaborative Development Environments", in Advances in Computers, vol. 59, 2003, pp. 1–27. | 329 | 1. G. Booch, A. W. Brown, "Collaborative Development Environments", in Advances in Computers, vol. 59, 2003, pp. 1–27. |