Commit 6cc43c1e1158f7a146fdbcad52b28d5fcc859d34
Exists in
master
and in
3 other branches
Merge branch 'master' of http://softwarepublico.gov.br/gitlab/softwarepublico/articles
Showing
1 changed file
with
65 additions
and
70 deletions
Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
@@ -3,99 +3,94 @@ | @@ -3,99 +3,94 @@ | ||
3 | ## Abstract | 3 | ## Abstract |
4 | 4 | ||
5 | For many software development teams, the first things that come to mind | 5 | For many software development teams, the first things that come to mind |
6 | -regarding Continuous Delivery (CD) are the operational aspects and the competitive | ||
7 | -benefits. In our experience, it was much more: it was a survival technique. This article | ||
8 | -presents how we applied CD in a Brazilian Government project for the | ||
9 | -development of a Collaborative Development Environment (CDE), sharing its | ||
10 | -unconventional challenges and the strategies used to overcome them. This report | ||
11 | -from the trench of the Brazilian Federal Government can help practitioners to | ||
12 | -understand how important CD adoption is to their projects. | 6 | +regarding Continuous Delivery (CD) are the operational aspects and the |
7 | +competitive benefits. In our experience, it was much more: it was a survival | ||
8 | +technique. This article presents how we applied CD in a Brazilian Government | ||
9 | +project for the development of a Collaborative Development Environment (CDE), | ||
10 | +sharing its unconventional challenges and the strategies used to overcome them. | ||
11 | +This report from the trench of the Brazilian Federal Government can help | ||
12 | +practitioners to understand how important CD adoption is to their projects. | ||
13 | 13 | ||
14 | ## Introduction and Context | 14 | ## Introduction and Context |
15 | 15 | ||
16 | We worked on a three-year-long Brazilian government project to evolve an | 16 | We worked on a three-year-long Brazilian government project to evolve an |
17 | -existing platform that had technical issues and lacked political support. The | ||
18 | -evolution project started in a presidential election year and everyone involved | ||
19 | -were under pressure to show results. Even with the re-election of the Brazilian | ||
20 | -President in 2014, leaderships in governmental agencies ended up changing. Each | ||
21 | -one of them had different political agendas which affected the project's | ||
22 | -requirements previously approved. Despite this scenario of instability, we | ||
23 | -managed to avoid delays in the delivery schedule and to prevent the project's | ||
24 | -cancellation. | ||
25 | - | ||
26 | -Our team from the University of Brasília (UnB) and the University of São Paulo | 17 | +existing platform that had technical issues and lacked political support. Our |
18 | +team from the University of Brasília (UnB) and the University of São Paulo | ||
27 | (USP) developed the new platform for the Brazilian Public Software (SPB, | 19 | (USP) developed the new platform for the Brazilian Public Software (SPB, |
28 | -Portuguese acronym) Portal (www.softwarepublico.gov.br) from 2014 to 2016. The | ||
29 | -SPB Portal evolved to a Collaborative Development Environment [1] and this | ||
30 | -evolution brought about important benefits not just to the Brazilian | ||
31 | -government, but also to society as a whole. For the government, the | ||
32 | -bureaucracy on using the same software across governmental agencies, duplicate | ||
33 | -works, and costs all were reduced. The society gained a transparency and | ||
34 | -collaboration mechanism, since anyone can check the government expenses on | ||
35 | -software and contribute to the software communities. To achieve these goals, | ||
36 | -rather than writing everything from scratch, we decided to integrate several | ||
37 | -free software tools such as Gitlab (www.gitlab.com), Mailman | 20 | +Portuguese acronym) Portal from 2014 to 2016. The SPB Portal |
21 | +(www.softwarepublico.gov.br) evolved to a Collaborative Development Environment | ||
22 | +[1] and this evolution brought important benefits not just to the Brazilian | ||
23 | +government, but also to society as a whole. For the government, the bureaucracy | ||
24 | +of using the same software across governmental agencies and the cost of | ||
25 | +developing similar software projects were all reduced. The society gained a | ||
26 | +transparent and collaborative mechanism, since anyone can check the government | ||
27 | +expenses on software and contribute to project communities. To achieve these | ||
28 | +goals, rather than writing everything from scratch, we decided to integrate | ||
29 | +several free software tools such as Gitlab (www.gitlab.com), Mailman | ||
38 | (www.gnu.org/software/mailman), Noosfero (www.noosfero.org), and Colab | 30 | (www.gnu.org/software/mailman), Noosfero (www.noosfero.org), and Colab |
39 | (www.github.com/colab). | 31 | (www.github.com/colab). |
40 | 32 | ||
41 | -During the entire SPB Portal evolution project, we had to handle three distinct | ||
42 | -issues, usual in a software engineering scenario: achieving the goals which have | ||
43 | -guided the platform development, managing the diversity of team project | ||
44 | -members, and communicating effectively with clients (in this case, | ||
45 | -government agents). Handling the interaction of these elements was not easy and | ||
46 | -the unstable Brazilian political scenario only made things worse. | 33 | +The project started in a presidential election year and everyone involved was |
34 | +under pressure to show results. Even with the re-election of the Brazilian | ||
35 | +President in 2014, leaderships in governmental agencies ended up changing. Each | ||
36 | +one of them had different political agendas which affected the project's | ||
37 | +requirements previously approved. Besides that scenario of instability, we | ||
38 | +overcame three distinct issues: (i) achieving the goals which have guided the | ||
39 | +platform development, (ii) managing the diversity of team members, and (iii) | ||
40 | +communicating effectively with government agents (our "clients"). Handling the | ||
41 | +interaction of these elements was challenging and the unstable Brazilian | ||
42 | +political scenario only made things worse. | ||
47 | 43 | ||
48 | To achieve the SPB project goals, we had to overcome strong political bias tied | 44 | To achieve the SPB project goals, we had to overcome strong political bias tied |
49 | -with complicated technical issues and relatively low budget. Because it | ||
50 | -is open to the public, the government representatives saw the platform as a | ||
51 | -marketing opportunity and sometimes ignored technical advice in favor of | 45 | +with complicated technical issues and relatively low budget. Because the |
46 | +project is open to the public, the government representatives saw the platform | ||
47 | +as a marketing opportunity and sometimes ignored technical advice in favor of | ||
52 | political decisions. Furthermore, integrating a number of distinct systems to | 48 | political decisions. Furthermore, integrating a number of distinct systems to |
53 | work seamlessly was not an easy job. We had to learn how each system worked and | 49 | work seamlessly was not an easy job. We had to learn how each system worked and |
54 | to come up with ideas of how to integrate them as fast as possible, with a team | 50 | to come up with ideas of how to integrate them as fast as possible, with a team |
55 | of mostly inexperienced developers. | 51 | of mostly inexperienced developers. |
56 | 52 | ||
57 | -We also had to manage the diversity of the SPB team members. This team was | ||
58 | -composed of approximately 50 undergraduate students (not all simultaneously) | ||
59 | -together with professors, masters students, professional designers and | ||
60 | -senior developers from the Brazilian free software community. Undergraduate | ||
61 | -students received a scholarship and, for most of them, this R&D project was | ||
62 | -their first professional experience. Seniors developers and masters students | ||
63 | -had two important contributions to the project: transfer knowledge to | ||
64 | -undergraduate students and address hard tasks. Finally, professors were | ||
65 | -responsible for interacting with the Brazilian government and controlling the | ||
66 | -political pressures applied to the project. | ||
67 | - | ||
68 | -The third point to be handled was the communication with two independent groups | ||
69 | -of government representatives: requirements analysts and deployment | ||
70 | -technicians. Requirements analysts were the real representatives of the | ||
71 | -Brazilian government in the project and their role was to test new | ||
72 | -features, to provide feedback to the development team, and to report for the | ||
73 | -government leaders. Deployment technicians were in charge of administering the host | ||
74 | -machines wherein SPB platform was running. They were, theoretically, | ||
75 | -responsible for deploying the project. However, the new SPB Portal was composed | ||
76 | -of more than ten different software projects, generating a complex deployment | ||
77 | -process, for instance, working on seven servers. | 53 | +We also had to manage the diversity of our team members. The team was composed |
54 | +of approximately 50 undergraduate students (not all simultaneously) together | ||
55 | +with professors, master students, professional designers and senior developers | ||
56 | +from the Brazilian free software community. Undergraduate students received a | ||
57 | +scholarship and, for most of them, this R&D project was their first | ||
58 | +professional experience. Senior developers and master students had two | ||
59 | +important contributions to the project: transfer knowledge to undergraduate | ||
60 | +students and more complex tasks. Finally, professors were responsible for | ||
61 | +interacting with the Brazilian government and controlling political pressures | ||
62 | +applied to the project. | ||
63 | + | ||
64 | +Moreover, we needed to communicate with two independent groups of government | ||
65 | +representatives. Requirement analysts were real representatives of the | ||
66 | +Brazilian government in the project and their role was to test new features, to | ||
67 | +provide feedback to the development team, and to report to government leaders. | ||
68 | +Deployment technicians were in charge of managing the host machines wherein SPB | ||
69 | +platform was running. They were, theoretically, responsible for deploying the | ||
70 | +project. However, the new SPB Portal was composed of more than ten different | ||
71 | +software projects, working on seven servers, generating a complex deployment | ||
72 | +process. | ||
78 | 73 | ||
79 | To handle the interaction between these three elements, we realised we needed | 74 | To handle the interaction between these three elements, we realised we needed |
80 | to take control over the deployment process. We used CD as a mean to fulfill | 75 | to take control over the deployment process. We used CD as a mean to fulfill |
81 | the government expectations and to provide quick response to their requests, | 76 | the government expectations and to provide quick response to their requests, |
82 | which were influenced most of the time by the uncertainties of the project's | 77 | which were influenced most of the time by the uncertainties of the project's |
83 | continuity. We believed we would keep the project alive, even in a politically | 78 | continuity. We believed we would keep the project alive, even in a politically |
84 | -unstable and technically complex scenario. For this reason, we worked hard to | ||
85 | -automate the deploy process; for instance, one of our senior developers created | ||
86 | -a Chef-Server (www.chef.io/chef) front-end tool called Chake | ||
87 | -(www.gitlab.com/terceiro/chake) to help us to manage the multiple hosts needed | ||
88 | -for the project. We also formed a specific team dedicated to the deployment | 79 | +unstable and technically complex scenario. For this reason, we focused on |
80 | +automating automate the deploy process; for instance, one of our senior | ||
81 | +developers created a Chef-Server (www.chef.io/chef) front-end tool called Chake | ||
82 | +(www.gitlab.com/terceiro/chake) to help us manage the multiple hosts needed for | ||
83 | +the project. We also formed a specific team dedicated to the deployment | ||
89 | process. This team was responsible for maturing our CD pipeline, giving us | 84 | process. This team was responsible for maturing our CD pipeline, giving us |
90 | confidence and agility to comply with government requirements. | 85 | confidence and agility to comply with government requirements. |
91 | 86 | ||
92 | In this report, we describe our CD pipeline and how it improved our delivery | 87 | In this report, we describe our CD pipeline and how it improved our delivery |
93 | -performance. The CD made us adaptable for all requested changes and helped us | ||
94 | -mitigate those aforementioned political challenges, besides the technical | ||
95 | -issues. Among CD’s known benefits [2], we also explain those which were the most | ||
96 | -important in the project scenario for us: improving customer satisfaction and | ||
97 | -making reliable releases. Both kept the project alive for more two years during | ||
98 | -the worst political crisis after the re-democratization in Brazil. | 88 | +performance. Continuous Delivery made us adaptable for all requested changes |
89 | +and helped us mitigate those aforementioned political challenges, besides the | ||
90 | +technical issues. Among CD’s known benefits [2] we also explain those which | ||
91 | +were the most important in our scenario: improving customer satisfaction and | ||
92 | +making reliable releases. Both kept the project alive for years during the | ||
93 | +worst political crisis after the re-democratization in Brazil. | ||
99 | 94 | ||
100 | ## Our Continuous Delivery Pipeline | 95 | ## Our Continuous Delivery Pipeline |
101 | 96 | ||
@@ -211,7 +206,7 @@ jobs. | @@ -211,7 +206,7 @@ jobs. | ||
211 | 206 | ||
212 | ### Shared responsibility | 207 | ### Shared responsibility |
213 | 208 | ||
214 | -Before the adoption of the CD, the development team could not track what happened to the code | 209 | +Before the adoption of CD, the development team could not track what happened to the code |
215 | after its delivery, since government technicians were the only responsibles | 210 | after its delivery, since government technicians were the only responsibles |
216 | for deploying the project. The implementation of the referred | 211 | for deploying the project. The implementation of the referred |
217 | approach influenced developers on taking ownership of the project because it | 212 | approach influenced developers on taking ownership of the project because it |
@@ -255,7 +250,7 @@ validated functionalities and sent feedback to developers, patches were | @@ -255,7 +250,7 @@ validated functionalities and sent feedback to developers, patches were | ||
255 | developed and new versions were packaged and deployed to the VE quickly, | 250 | developed and new versions were packaged and deployed to the VE quickly, |
256 | steadily, and reliably. The constant monitoring of the development work brought | 251 | steadily, and reliably. The constant monitoring of the development work brought |
257 | greater security to the governmental nucleus and improved the interactions of | 252 | greater security to the governmental nucleus and improved the interactions of |
258 | -this with the team of developers. | 253 | +this with our development team. |
259 | 254 | ||
260 | 255 | ||
261 | ## Challenges | 256 | ## Challenges |