Commit 5f1b08cd57e1015532050626d75567e5472a7ab2

Authored by Paulo Meireles
1 parent 112263f9

[ieeeSW] General review

Showing 1 changed file with 199 additions and 213 deletions   Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
@@ -30,7 +30,7 @@ source software development. Contact her at melissa.srw@gmail.com . @@ -30,7 +30,7 @@ source software development. Contact her at melissa.srw@gmail.com .
30 30
31 __Paulo Meirelles__ received his Ph.D. in Computer Science from the Institute of 31 __Paulo Meirelles__ received his Ph.D. in Computer Science from the Institute of
32 Mathematics and Statistics at the University of São Paulo. He is full-time 32 Mathematics and Statistics at the University of São Paulo. He is full-time
33 -Professor at the University of Brasilia, and coordinated the new SPB Portal 33 +Professor at the University of Brasilia, and coordinated the new SPB portal
34 project. His research interest areas are: Free Software, Agile methods, Static 34 project. His research interest areas are: Free Software, Agile methods, Static
35 analysis, and Source code metrics. Contact him at paulormm@ime.usp.br. 35 analysis, and Source code metrics. Contact him at paulormm@ime.usp.br.
36 36
@@ -42,9 +42,9 @@ him at fabio.kon@ime.usp.br. @@ -42,9 +42,9 @@ him at fabio.kon@ime.usp.br.
42 42
43 For many software development teams, the first aspects that come to mind 43 For many software development teams, the first aspects that come to mind
44 regarding Continuous Delivery (CD) are the operational challenges and the 44 regarding Continuous Delivery (CD) are the operational challenges and the
45 -competitive benefits. In our experience, the CD was much more: it was a survival  
46 -technique. This article presents how and why we applied CD in a Brazilian  
47 -government project for the development of a Collaborative Development 45 +competitive benefits. In our experience, the CD was much more: it was a
  46 +survival technique. This article presents how and why we applied CD in a
  47 +Brazilian government project for the development of a Collaborative Development
48 Environment (CDE), sharing the challenges we faced and the strategies used to 48 Environment (CDE), sharing the challenges we faced and the strategies used to
49 overcome them. 49 overcome them.
50 50
@@ -56,29 +56,29 @@ Siqueira/Paulo: Ao meu ver, a introdução deixa bem claro o nosso "punch" e o r @@ -56,29 +56,29 @@ Siqueira/Paulo: Ao meu ver, a introdução deixa bem claro o nosso "punch" e o r
56 --> 56 -->
57 57
58 We worked on a three-year-long Brazilian government project to evolve an 58 We worked on a three-year-long Brazilian government project to evolve an
59 -existing platform that had technical issues and lacked political support.  
60 -This project, started in 2014, was a partnership between the Ministry of  
61 -Planning, Budget, and Management (MP) and two public universities:  
62 -University of Brasília (UnB) and University of São Paulo (USP), to  
63 -modernize Brazilian Public Software (SPB) portal.  
64 -  
65 -With this partnership, the SPB Portal (www.softwarepublico.gov.br) evolved to a  
66 -Collaborative Development Environment[1], and this evolution brought significant  
67 -benefits not just to the Brazilian government, but also to society as a whole.  
68 -The government could minimize bureaucracy and costs of software development,  
69 -encouraging the use of the same set of applications across different government  
70 -agencies. The society also gained a mechanism of transparency and collaboration,  
71 -since anyone can check the government expenses on software and contribute to  
72 -project communities. 59 +existing platform that had technical issues and lacked political support. This
  60 +project, started in 2014, was a partnership between the Ministry of Planning,
  61 +Budget, and Management (MP) and two public universities: University of Brasília
  62 +(UnB) and University of São Paulo (USP), to modernize Brazilian Public Software
  63 +(SPB) portal.
  64 +
  65 +With this partnership, the SPB portal (www.softwarepublico.gov.br) evolved to a
  66 +Collaborative Development Environment[1], and this evolution brought
  67 +significant benefits not just to the Brazilian government, but also to society
  68 +as a whole. The government could minimize bureaucracy and costs of software
  69 +development, encouraging the use of the same set of applications across
  70 +different government agencies. The society also gained a mechanism of
  71 +transparency and collaboration, since anyone can check the government expenses
  72 +on software and contribute to project communities.
73 73
74 In this article, we discuss the use of Continuous Delivery (CD) during our 74 In this article, we discuss the use of Continuous Delivery (CD) during our
75 experience as the academic partner in this project. We focus on how we managed 75 experience as the academic partner in this project. We focus on how we managed
76 -to implement CD in a large institution with traditional values and how CD helped  
77 -to build trust between the government and the university development team. CD  
78 -enabled us to show our progress and earned the government’s confidence that we  
79 -could adequately fulfill their requests, becoming an essential aspect of our  
80 -interaction with them. According to this experience, the use of CD as a tool to  
81 -build such trust relationships is yet another of its benefits[2]. 76 +to implement CD in a large institution with traditional values and how CD
  77 +helped to build trust between the government and the university development
  78 +team. CD enabled us to show our progress and earned the government’s confidence
  79 +that we could adequately fulfill their requests, becoming an essential aspect
  80 +of our interaction with them. According to this experience, the use of CD as a
  81 +tool to build such trust relationships is yet another of its benefits[2].
82 82
83 ## Context 83 ## Context
84 <!--- 84 <!---
@@ -89,20 +89,21 @@ Paulo: O ponto não foi gov e não gov. Vamos esclarecer que o SPB foi algo part @@ -89,20 +89,21 @@ Paulo: O ponto não foi gov e não gov. Vamos esclarecer que o SPB foi algo part
89 --> 89 -->
90 90
91 The SPB is a governmental program of the MP created to foster sharing and 91 The SPB is a governmental program of the MP created to foster sharing and
92 -collaboration on Open Source Software (OSS) development for the Brazilian public  
93 -administration. For their projects, the MP managed both software requirements  
94 -and server infrastructure. However, its hierarchical and traditional processes  
95 -made them unfamiliar with new software development techniques, such as CD. Any  
96 -of our requests had to pass through layers of bureaucracy before being answered.  
97 -Requesting access to their infrastructure to make the deploy was not different. 92 +collaboration on Open Source Software (OSS) development for the Brazilian
  93 +public administration. For their projects, the MP managed both software
  94 +requirements and server infrastructure. However, its hierarchical and
  95 +traditional processes made them unfamiliar with new software development
  96 +techniques, such as CD. Any of our requests had to pass through layers of
  97 +bureaucracy before being answered. Requesting access to their infrastructure
  98 +to make the deploy was not different.
98 99
99 During its lifetime, the project suffered significant interference from the 100 During its lifetime, the project suffered significant interference from the
100 -board of directors because the portal represents an interface between government  
101 -and society. In light of political interests, directors continually imposed  
102 -changes to the platform while ignoring our technical advice. In 2015, the board  
103 -of directors was changed and, with it, the vision of the project. New directors  
104 -had different political agendas which affected the project's requirements  
105 -previously approved. 101 +board of directors because the portal represents an interface between
  102 +government and society. In light of political interests, directors continually
  103 +imposed changes to the platform while ignoring our technical advice. In 2015,
  104 +the board of directors was changed and, with it, the vision of the project. New
  105 +directors had different political agendas which affected the project's
  106 +requirements previously approved.
106 107
107 <!--- 108 <!---
108 Comment: 109 Comment:
@@ -116,8 +117,9 @@ Melissa: Reformular o enunciado do primeiro problema (FEITO por Siqueira e Diego @@ -116,8 +117,9 @@ Melissa: Reformular o enunciado do primeiro problema (FEITO por Siqueira e Diego
116 In this context, we overcame three distinct challenges: (1) find a system 117 In this context, we overcame three distinct challenges: (1) find a system
117 solution wherein government and development team agree, (2) deconstruct the 118 solution wherein government and development team agree, (2) deconstruct the
118 widespread belief among the government agents that any project in partnership 119 widespread belief among the government agents that any project in partnership
119 -with a University is doomed to fail, and (3) deal with bureaucracies involved in  
120 -the deployment process by the MP. 120 +with a University is doomed to fail, and (3) deal with bureaucracies involved
  121 +in the deployment process by the MP.
  122 +
121 <!--- 123 <!---
122 Comment: 124 Comment:
123 125
@@ -126,14 +128,14 @@ Comment: @@ -126,14 +128,14 @@ Comment:
126 Siqueira/Paulo: Contamos essa história sem ter que entrar em detalhes aqui. Contudo, será preciso fazer a seção de pipeline harmonizar com isso 128 Siqueira/Paulo: Contamos essa história sem ter que entrar em detalhes aqui. Contudo, será preciso fazer a seção de pipeline harmonizar com isso
127 --> 129 -->
128 130
129 -Firstly, to find a system solution wherein government and development team  
130 -agree, we designed the SPB Portal as a CDE with additional social features. Due 131 +Firstly, to find a system solution wherein MP agents and development team
  132 +agree, we designed the SPB portal as a CDE with additional social features. Due
131 to the complexity of creating such a system from scratch, we decided to adapt 133 to the complexity of creating such a system from scratch, we decided to adapt
132 -and integrate existing open source tools to build a system-of-systems [3]. We  
133 -created a solution that orchestrates software and allowed us to smoothly provide  
134 -a unified interface for final users, including single sign-on and global  
135 -searches [4]. On top of that, the new SPB Portal was an unprecedented platform  
136 -to the Brazilian government, with a complicated deployment process. 134 +and integrate existing OSS tools to build a system-of-systems [3]. We created a
  135 +solution that orchestrates software and allowed us to smoothly provide a
  136 +unified interface for final users, including single sign-on and global searches
  137 +[4]. On top of that, the new SPB portal was an unprecedented platform to the
  138 +Brazilian government, with a complicated deployment process.
137 139
138 <!--- 140 <!---
139 Comment: "Can the authors provide some data as well? As I said this is a unique scenario and the community can greatly benefit. For example, how many devs at any given time, how many features/unit time (month/year), how frequently were releases made, how frequently did you meet with the client, how frequently did the requirements change are some example questions to which if we had data, it would place the study in greater context." 141 Comment: "Can the authors provide some data as well? As I said this is a unique scenario and the community can greatly benefit. For example, how many devs at any given time, how many features/unit time (month/year), how frequently were releases made, how frequently did you meet with the client, how frequently did the requirements change are some example questions to which if we had data, it would place the study in greater context."
@@ -148,41 +150,25 @@ Comment: @@ -148,41 +150,25 @@ Comment:
148 Siqueira/Paulo: Isso aqui também fica resolvido 150 Siqueira/Paulo: Isso aqui também fica resolvido
149 --> 151 -->
150 152
151 -<!---  
152 -Paulo: este parágrafo abaixo ficou tem redundância com o segundo do contexto.  
153 -  
154 -Beyond these technical and organizational issues, we had to overcome strong political bias and relatively low budget. The project suffered significant interference from the board of directors because the SPB Portal represents an interface between government and society. In light of political interests, directors continually imposed changes to the platform while ignoring our technical advice. During 2015, Brazil faced a political crisis that impacted the SPB: the board of directors was changed twice and, with it, the vision of the project. New directors primarily focused on keeping their distance from previous administrations.  
155 --->  
156 -  
157 -<!---  
158 -Paulo: o segundo problema foi alterado em relação à primeira versão, mas a explicação não ficou muito diferente.  
159 -  
160 -Siqueira: trocar conhecimento técnico por interesse, disponibilidade ou esforço ...  
161 --->  
162 -  
163 -Secondly, we had to face the widespread belief among government agents that any  
164 -project in partnership with a University is doomed to fail. Our team was not  
165 -from a typical company, consisting mainly of undergraduate students coordinated  
166 -by two professors. At the first year, we had a group composed of 24  
167 -undergraduate students, one designer, and two senior developers. In 2015, our  
168 -team grew to 36 students, two designers, eight senior developers. At the end of  
169 -in project, due to budget constraint, we had 20 students, one designer, and two  
170 -developers. Also, on the government side, the SPB Portal evolution was the first  
171 -software development collaboration between university and government experienced  
172 -by the MP analysts involved in the project.  
173 -  
174 -<!---  
175 -Paulo: o terceiro problema é novo em relação à primeira versão, mas na prática tínhamos tratado a questão  
176 ---> 153 +Secondly, we had to face the widespread belief among MP agents that any project
  154 +in partnership with a University is doomed to fail. Our team was not from a
  155 +typical company, consisting mainly of undergraduate students coordinated by two
  156 +professors. At the first year, we had a group composed of 24 undergraduate
  157 +students, one designer, and two senior developers. In 2015, our team grew to 36
  158 +students, two designers, eight senior developers. At the end, due to budget
  159 +constraint, we had 20 students, one designer, and two developers. On the
  160 +government side, the SPB portal evolution was the first software development
  161 +collaboration between university and government experienced by the MP analysts
  162 +involved in the project.
177 163
178 Lastly, our team thought software deployment differently than the MP. On our 164 Lastly, our team thought software deployment differently than the MP. On our
179 side, we believe that frequent deliveries are better for the project’s success. 165 side, we believe that frequent deliveries are better for the project’s success.
180 However, the MP works with the idea of a single deployment at the end of the 166 However, the MP works with the idea of a single deployment at the end of the
181 project. In other words, neither the bureaucratic structure of the MP nor its 167 project. In other words, neither the bureaucratic structure of the MP nor its
182 -technical abilities were conducive to this style of work. Furthermore, there was  
183 -little effort to deploy new versions of the system promptly. That ended up  
184 -hampering the benefits of the tool and preventing us from showing off the fruits  
185 -of the project to those responsible for evaluating it. 168 +technical abilities were conducive to this style of work. Furthermore, there
  169 +was little effort to deploy new versions of the system promptly. That ended up
  170 +hampering the benefits of the tool and preventing us from showing off the
  171 +fruits of the project to those responsible for evaluating it.
186 172
187 <!--- 173 <!---
188 "the article is missing the ah,ha moment. Was there something interesting that you learned (could be a small thing) that you use to adapt the CI process when applying it in a gov or large org context versus a smaller org? " 174 "the article is missing the ah,ha moment. Was there something interesting that you learned (could be a small thing) that you use to adapt the CI process when applying it in a gov or large org context versus a smaller org? "
@@ -190,43 +176,45 @@ of the project to those responsible for evaluating it. @@ -190,43 +176,45 @@ of the project to those responsible for evaluating it.
190 Siqueira/Paulo: eis o AH-HA moment! 176 Siqueira/Paulo: eis o AH-HA moment!
191 --> 177 -->
192 178
193 -These challenges made our relationship with the MP tense, in particular at the  
194 -first year, and alerted us to the fact that they could finish the project at any  
195 -time. The deployment limitation was the substantial technical issue we could  
196 -tackle in the short term. As a result, we worked to deploy one version of the  
197 -project onto our infrastructure and showed it to the government evaluators. This  
198 -strategy proved them we could efficiently deliver new features, fulfill their  
199 -expectations regarding the delivery of the requirements, and incited them to  
200 -demand the last version working in the MP infrastructure. These results, in turn,  
201 -generated more pressure on the IT department responsible for the deployment  
202 -routines. With each CD cycle, we gradually built a new relationship among all  
203 -parties and, by the end of the project, we became active participants in the  
204 -deploy operations. 179 +These challenges made our relationship with the MP agents tense, in particular
  180 +at the first year, and alerted us to the fact that they could finish the
  181 +project at any time. The deployment limitation was the substantial technical
  182 +issue we could tackle in the short term. As a result, we worked to deploy one
  183 +version of the project onto our infrastructure and showed it to the government
  184 +evaluators. This strategy proved them we could efficiently deliver new
  185 +features, fulfill their expectations regarding the delivery of the
  186 +requirements, and incited them to demand the last version working in the MP
  187 +infrastructure. These results, in turn, generated more pressure on the IT
  188 +department responsible for the deployment routines. With each CD cycle, we
  189 +gradually built a new relationship among all parties and, by the end of the
  190 +project, we became active participants in the deploy operations.
205 191
206 ## Our Continuous Delivery Pipeline 192 ## Our Continuous Delivery Pipeline
207 193
208 -Figure 1 represents our CD pipeline. A new feature might require changes on more  
209 -than one SPB integrated software project. Notice that each one of them could be  
210 -modified independently. The pipeline started when new code arrived. As it went  
211 -through each step, it was tested and improved until it finally reached the  
212 -production environment. At this point we would restart the pipeline to release  
213 -more new code.  
214 -  
215 ![Deployment Pipeline](figures/pipeline_3.png) 194 ![Deployment Pipeline](figures/pipeline_3.png)
216 195
  196 +Figure 1 represents our CD pipeline. A new feature might require changes on
  197 +more than one SPB integrated software project. Notice that each one of them
  198 +could be modified independently. The pipeline started when new code arrived. As
  199 +it went through each step, it was tested and improved until it finally reached
  200 +the production environment. At this point we would restart the pipeline to
  201 +release more new code.
  202 +
217 ### Automated Tests 203 ### Automated Tests
218 204
219 <!--- 205 <!---
220 Diego: Deixei a mini explicação usando o Colab porque achei que ainda faz sentido 206 Diego: Deixei a mini explicação usando o Colab porque achei que ainda faz sentido
221 e está bem colocada. Se decidirmos tirar, teremos que repensar o parágrafo. 207 e está bem colocada. Se decidirmos tirar, teremos que repensar o parágrafo.
222 --> 208 -->
223 -The SPB portal is a system-of-systems with several integrated software projects.  
224 -Each of them, as well as the entire platform, had to be tested. These software  
225 -components have own test suite. Colab, a systems integration platform for web  
226 -applications based on a plugins architecture, orchestrate communication between  
227 -all of them. Therefore, we developed specific plugins for each portal software  
228 -component, such as Gitlab (www.gitlab.com) and Noosfero (www.noosfero.org). Each  
229 -plugin also has own test suite, and this set also worked as integration tests. 209 +
  210 +The SPB portal is a system-of-systems with several integrated software
  211 +projects. Each of them, as well as the entire platform, had to be tested.
  212 +These software components have own test suite. Colab (www.github.com/colab), a
  213 +systems integration platform for web applications based on a plugins
  214 +architecture, orchestrate communication between all of them. Therefore, we
  215 +developed specific plugins for each portal software component, such as Gitlab
  216 +(www.gitlab.com) and Noosfero (www.noosfero.org). Each plugin also has own test
  217 +suite, and this set also worked as integration tests.
230 218
231 Both unit and integration tests provided us the performance and security needed 219 Both unit and integration tests provided us the performance and security needed
232 to guarantee the stability of components and the platform. If any test suite 220 to guarantee the stability of components and the platform. If any test suite
@@ -236,13 +224,13 @@ step of preparing the release. @@ -236,13 +224,13 @@ step of preparing the release.
236 224
237 ### Preparing a New Release 225 ### Preparing a New Release
238 226
239 -An SPB Portal release was composed of all its software components releases. Each  
240 -software component release was a git tag that referred to a specific feature or  
241 -bug fix. When all tests passed for a given component, we manually created a new  
242 -tag for it. Therefore, a new tag on any software component yielded a new SPB  
243 -Portal release. More precisely, SPB had a script that produced a single release  
244 -for the entire system based on each component tag. At the end of this process,  
245 -we started packaging. 227 +An SPB portal release was composed of all its software components releases.
  228 +Each software component release was a Git tag that referred to a specific
  229 +feature or bug fix. When all tests passed for a given component, we manually
  230 +created a new tag for it. Therefore, a new tag on any software component
  231 +yielded a new SPB portal release. More precisely, SPB had a script that
  232 +produced a single release for the entire system based on each component tag. At
  233 +the end of this process, we started packaging.
246 234
247 ### Packaging 235 ### Packaging
248 236
@@ -254,47 +242,48 @@ repository. @@ -254,47 +242,48 @@ repository.
254 We decided to create own packages for each software component for the following 242 We decided to create own packages for each software component for the following
255 five reasons: 243 five reasons:
256 244
257 -* Not all software was packaged by the community and those that existed were outdated;  
258 -* Packaging makes it easy to manage the software on a given distribution;  
259 -* Packaging simplifies the deployment;  
260 -* Packaging follows the distribution's best practices and, 245 +1. Not all software was packaged by the community and those that existed were outdated;
  246 +1. Packaging makes it easy to manage the software on a given distribution;
  247 +1. Packaging simplifies the deployment;
  248 +1. Packaging follows the distribution's best practices and,
261 * Packaging allows configurations and permissions control. 249 * Packaging allows configurations and permissions control.
262 250
263 -After creating a new tag for one component, the developers informed the DevOps 251 +After creating a new tag for one component, the developers informed our DevOps
264 [5] team, and the packaging process began. A set of scripts fully automated the 252 [5] team, and the packaging process began. A set of scripts fully automated the
265 three packaging steps aforementioned. With all them running successfully, the 253 three packaging steps aforementioned. With all them running successfully, the
266 new packages would be ready to be used by our deployment scripts. 254 new packages would be ready to be used by our deployment scripts.
267 255
268 ### Validation Environment Deployment 256 ### Validation Environment Deployment
269 257
270 -The Validation Environment (VE) is a replica of the Production Environment (PE),  
271 -with two exceptions: only the government officers and project leaders had access  
272 -to it and all the data became anonymous. To configure the environment, we used a  
273 -configuration management tool named Chef with Chake support (serverless  
274 -configuration for Chef). It maintained environment consistency simplifying the  
275 -deployment process. Additionally, the packages we built on the last step were  
276 -readily available to be used by the management tool.  
277 -  
278 -Government agents used the VE to validate new features and required changes. Also,  
279 -the VE was used to verify the integrity of the entire portal as part of the next 258 +The Validation Environment (VE) is a replica of the Production Environment
  259 +(PE), with two exceptions: only the government officers and project leaders had
  260 +access to it and all the data became anonymous. To configure the environment,
  261 +we used a configuration management tool named Chef with Chake support (a
  262 +serverless configuration for Chef tool created by our team). It maintained
  263 +environment consistency simplifying the deployment process. Additionally, the
  264 +packages we built on the last step were readily available to be used by the
  265 +management tool.
  266 +
  267 +The MP agents used the VE to validate new features and required changes. The VE
  268 +also was used to verify the integrity of the entire portal as part of the next
280 step in the pipeline. 269 step in the pipeline.
281 270
282 ### Acceptance Tests 271 ### Acceptance Tests
283 272
284 -After we deployed a new SPB Portal version in the VE, the government  
285 -agents were responsible for checking features and bug fixes required by them. If  
286 -the requirement analysts identified a problem, they would notify the developers via  
287 -comments on the SPB Portal's issue tracker. The development team fixed the problem  
288 -and the pipeline restarted from scratch. If everything was validated, we moved forward. 273 +After we deployed a new SPB portal version in the VE, the MP agents were
  274 +responsible for checking features and bug fixes required by them. If the
  275 +requirement analysts identified a problem, they would notify the developers via
  276 +comments on the SPB portal's issue tracker. The development team fixed the
  277 +problem and the pipeline restarted from scratch. If everything was validated,
  278 +we moved forward.
289 279
290 ### Production Environment Deployment 280 ### Production Environment Deployment
291 281
292 -When the government finished the VE check, we could finally begin the deployment  
293 -in the Production Environment (PE). For this we also used our configuration  
294 -management tool, the same scripts and package versions as in the VE. After the  
295 -deploy was completed, both VE and PE were running identical software. Here was  
296 -the point where new features and bug fixes were finally available to the end  
297 -users. 282 +When the MP agents finished the VE check, we could finally begin the deployment
  283 +in the PE. For this we also used our configuration management tool, the same
  284 +scripts and package versions as in the VE. After the deploy was completed, both
  285 +VE and PE were running identical software. Here was the point where new
  286 +features and bug fixes were finally available to the end users.
298 287
299 ## Benefits 288 ## Benefits
300 289
@@ -306,26 +295,24 @@ Working with the government, we noticed the following additional benefits. @@ -306,26 +295,24 @@ Working with the government, we noticed the following additional benefits.
306 ### Strengthening Trust in Work Relationship with the Government 295 ### Strengthening Trust in Work Relationship with the Government
307 296
308 Continuous delivery was also a tool that helped to strengthen trust in the 297 Continuous delivery was also a tool that helped to strengthen trust in the
309 -relationship between developers and government analysts, as well as between the  
310 -analysts group and its superiors. Before using CD, analysts had access to the  
311 -features developed only at the end of the release, usually every four months.  
312 -However, this periodicity did not meet the requirements of their leaders, who  
313 -demanded monthly reports on the progress of the project. 298 +relationship between developers and MP agents. Before using CD, the MP analysts
  299 +had access to the features developed only at the end of the release, usually
  300 +every four months.
314 301
315 With the implementation of CD, intermediate and candidate versions became 302 With the implementation of CD, intermediate and candidate versions became
316 -available, allowing analysts to perform small validations over time. The  
317 -constant monitoring of the development work brought greater security to the  
318 -governmental nucleus and improved the interactions with our development team. 303 +available, allowing the MP analysts to perform small validations over time. The
  304 +constant monitoring of the development work brought greater security to the MP
  305 +leaders and improved the interactions with our development team.
319 306
320 ### Responsiveness to Change 307 ### Responsiveness to Change
321 308
322 Responsiveness was one of the direct benefits of adopting the CD pipeline. The 309 Responsiveness was one of the direct benefits of adopting the CD pipeline. The
323 -ability to react quickly to changes requested by the government was vital for  
324 -the renewal of the project over the years. Every meeting with the government  
325 -leader resulted in requirements and priorities changes, most of them motivated  
326 -by political needs. We believed that if we took too long to attend their  
327 -demands, the government would use undelivered requirements as a means to  
328 -justify the lack of financial support and the end of the project. 310 +ability to react quickly to changes requested by the MP agents was vital for
  311 +the renewal of the project over the years. Every meeting with the MP leaders
  312 +resulted in requirements and priorities changes, several of them motivated by
  313 +political needs. We observed that if we took too long to attend their demands,
  314 +the MP would use undelivered requirements as a means to justify the lack of
  315 +financial support and the end of the project.
329 316
330 CD helped us keep the production environment up-to-date, even with partial 317 CD helped us keep the production environment up-to-date, even with partial
331 versions of a feature. That way, we always had something to show on meetings, 318 versions of a feature. That way, we always had something to show on meetings,
@@ -336,61 +323,61 @@ would not go looking for other jobs. @@ -336,61 +323,61 @@ would not go looking for other jobs.
336 ### Shared Responsibility 323 ### Shared Responsibility
337 324
338 According to the MP process, the development team could not track what happened 325 According to the MP process, the development team could not track what happened
339 -to the code after its delivery, since government technicians were the only ones  
340 -responsible for deployment. The implementation of CD made developers feel  
341 -equally responsible for what was getting into production and take ownership of  
342 -the project.  
343 -%  
344 -Interestingly, the CD pipeline had the same effect on the team of requirement  
345 -analysts. They became more engaged in the whole process, opening and discussing  
346 -issues during the platform evolution. Additionally, developers worked to improve  
347 -the CD pipeline to speed up the process of making new features available in the  
348 -production environment for analysts’ validation. 326 +to the code after its delivery, since, theoretically, the MP technicians were
  327 +the only ones responsible for deployment. The implementation of CD made our
  328 +development team feel equally responsible for what was getting into production
  329 +and take ownership of the project.
  330 +
  331 +Interestingly, the CD pipeline had the same effect on the MP analysts. They
  332 +became more engaged in the whole process, opening and discussing issues during
  333 +the platform evolution. Additionally, developers worked to improve the CD
  334 +pipeline to speed up the process of making new features available in the
  335 +production environment for the MP analysts' validation.
349 336
350 337
351 ### Synchronicity Between Government and Development 338 ### Synchronicity Between Government and Development
352 339
353 Despite the positive impacts that the CD pipeline brought to the project, its 340 Despite the positive impacts that the CD pipeline brought to the project, its
354 implementation was not smooth at first. The CD pipeline performance depended on 341 implementation was not smooth at first. The CD pipeline performance depended on
355 -the synchronicity between developers and government analysts so that the 342 +the synchronicity between our development team and the MP agents so that the
356 latter were prepared to start a step as soon as the former concluded the 343 latter were prepared to start a step as soon as the former concluded the
357 -previous step and vice versa. Initially, the agenda of the government team  
358 -did not contemplate this concern, which generated delays in the validation  
359 -of new features. This situation combined with governmental bureaucracy  
360 -(up to 3 days) to release access to the production environment resulted in  
361 -additional delays for the deployment step to begin. This problem was softened  
362 -when the analysts realized the impact of these delays on the final product and  
363 -decided to allocate the revisions in its work schedule and to request the access  
364 -to production in time. 344 +previous step and vice versa. Initially, the agenda of the MP agents did not
  345 +contemplate this concern, which generated delays in the validation of new
  346 +features. This situation combined with governmental bureaucracy (up to 3 days)
  347 +to release access to the production environment resulted in additional delays
  348 +for the deployment step to begin. This problem was softened when the MP
  349 +analysts realized the impact of these delays on the final product and decided
  350 +to allocate the revisions in its work schedule and to request the access to
  351 +production in time.
365 352
366 ## Challenges 353 ## Challenges
367 354
368 -Due to the successful building of the CD pipeline, we improved the government  
369 -deployment process and kept the project alive in an unstable political scenario.  
370 -We map here lessons learned in this successful case, as well as issues that  
371 -future works between industry and academia still need to address. 355 +Due to the successful building of the CD pipeline, we improved the MP
  356 +deployment process and kept the project alive. We map here lessons learned in
  357 +this successful case, as well as issues that future works between industry and
  358 +academia still need to address.
372 359
373 ### Build CD From Scratch 360 ### Build CD From Scratch
374 361
375 Taking on responsibilities for implementing CD impacted on the whole team. 362 Taking on responsibilities for implementing CD impacted on the whole team.
376 -Our team did not have know-how in this approach, and we had few working hours  
377 -available to allocate for the building of the pipeline. The construction and  
378 -maintenance of the CD process were possible by taking some decisions to mature  
379 -the project: 363 +Mostly, our team members did not have know-how in this approach, and we had few
  364 +working hours available to allocate for the building of the pipeline. The
  365 +construction and maintenance of the CD process were possible by taking some
  366 +decisions to mature the project:
380 367
381 -1. _Select the most experienced professionals and some developers of the  
382 -project to work on a specific team for DevOps._ These professionals used their  
383 -experiences in OSS projects to get an initial proposal for the deployment process.  
384 -The solution enabled us to automate the deployment, even though the process was  
385 -still rudimentary. 368 +1. _Select the most experienced senior developers and some advanced software
  369 +engineering students of the project to work on a specific team for DevOps._
  370 +These senior developers used their experiences in OSS projects to get an
  371 +initial proposal for the deployment process. The solution enabled us to
  372 +automate the deployment, even though the process was still rudimentary.
386 373
387 2. _Interchange team members and encouraging teammates to migrate to DevOps 374 2. _Interchange team members and encouraging teammates to migrate to DevOps
388 team._ The benefits of these movements were twofold: mitigating the difficulty 375 team._ The benefits of these movements were twofold: mitigating the difficulty
389 to pass the knowledge between DevOps developers and features developers, and 376 to pass the knowledge between DevOps developers and features developers, and
390 evolving the process on-the-fly. 377 evolving the process on-the-fly.
391 378
392 -Building a CD pipeline was hard in the beginning. We believe that more tools  
393 -to provide out-of-the-box standardized CD pipelines would be of great help for 379 +Building a CD pipeline was hard in the beginning. We believe that more tools to
  380 +provide out-of-the-box standardized CD pipelines would be of great help for
394 inexperienced teams. Tools that track each step of the pipeline and organize 381 inexperienced teams. Tools that track each step of the pipeline and organize
395 logs in a human-manageable way are necessary too. We also suggest further 382 logs in a human-manageable way are necessary too. We also suggest further
396 research on how to effectively spread knowledge across inexperienced developers 383 research on how to effectively spread knowledge across inexperienced developers
@@ -398,47 +385,48 @@ in a high turnover scenario. @@ -398,47 +385,48 @@ in a high turnover scenario.
398 385
399 ### Overcoming Mistrust 386 ### Overcoming Mistrust
400 387
401 -Taking an unfamiliar approach requires trust. In the Ministry of Planning,  
402 -software is the product delivered at the end of a development contract. They  
403 -expected and were prepared to validate and deploy a single delivery. Because  
404 -the SPB portal is a system of systems, the steady growth of its complexity made  
405 -large deliveries unsustainable. Also, the long time for homologation of  
406 -developed features gave the government room to change requirements and priorities.  
407 -The CD approach was necessary, but how to build trust and gain autonomy to  
408 -implement a process that was not yet part of the dynamics of the Ministry? 388 +Taking an unfamiliar approach requires trust. In the MP, software is the
  389 +product delivered at the end of a development contract. They expected and were
  390 +prepared to validate and deploy a single delivery. Because the SPB portal is a
  391 +system-of-systems, the steady growth of its complexity made large deliveries
  392 +unsustainable. The long time for homologation of developed features also gave
  393 +the government room to change requirements and priorities. The CD approach was
  394 +necessary, but how to build trust and gain autonomy to implement a process that
  395 +was not yet part of the dynamics of the Ministry?
409 396
410 1. _Demonstrate actual results, do not simply tell._ Initially, we did not have 397 1. _Demonstrate actual results, do not simply tell._ Initially, we did not have
411 -access to the government infrastructure, so we created our own validation  
412 -environment. Thus, we were able to follow the CD pipeline until the stage of  
413 -production deployment, when we faced two problems. Our pace of intermediate  
414 -deliveries to the government was faster than the deployment in production by their  
415 -technicians. Furthermore, specific issues of government infrastructure made some  
416 -validated features not work as expected in the production environment. That  
417 -situation gave us arguments to negotiate access to production.  
418 -  
419 -2. _Make our project management transparent and collaborative for analysts._  
420 -Allowing the analysts to follow our process for version deliveries and bug  
421 -fixes, we showed them we were meeting our commitments. They started to interact  
422 -more actively in the generation of versions and became part of the  
423 -process. After understanding the process, analysts helped us in negotiations  
424 -with government agents. Finally, the agents created a VE as an isolated replica  
425 -of PE and gave us access to it. 398 +access to the MP infrastructure, so we created our own validation environment.
  399 +Thus, we were able to follow the CD pipeline until the stage of production
  400 +deployment, when we faced two problems. Our pace of intermediate deliveries to
  401 +the government was faster than the deployment in production by the MP
  402 +technicians. Furthermore, specific issues of the MP infrastructure made some
  403 +validated features not work as expected in the PE. That situation gave us
  404 +arguments to negotiate access to PE.
  405 +
  406 +2. _Make our project management transparent and collaborative for the MP
  407 +agents._ Allowing the MP analysts to follow our process for version deliveries
  408 +and bug fixes, we showed them we were meeting our commitments. They started to
  409 +interact more actively in the generation of versions and became part of the
  410 +process. After understanding the process, the MP analysts helped us in
  411 +negotiations with the MP leaders. Finally, the MP technicians created a VE as
  412 +an isolated replica of PE and gave us access to it.
426 413
427 3. _Gain the confidence of government agents._ With the replica of PE, we were 414 3. _Gain the confidence of government agents._ With the replica of PE, we were
428 -able to run the entire pipeline and won the trust of the government analysts and 415 +able to run the entire pipeline and won the trust of the MP analysts and
429 technicians involved in the process. On the one hand, analysts saw the 416 technicians involved in the process. On the one hand, analysts saw the
430 -mobilization and responsiveness of the team to generate a new version package. 417 +mobilization and responsiveness of our team to generate a new version package.
431 On the other hand, technicians recognized the quality of our packages and our 418 On the other hand, technicians recognized the quality of our packages and our
432 -deployment process. Finally, the government agents then realized that it could  
433 -be beneficial for the project if they granted us access to part of the  
434 -infrastructure. 419 +deployment process. Finally, the MP agents then realized that it could be
  420 +beneficial for the project if they granted us access to the infrastructure,
  421 +included the PE.
435 422
436 More research is required on developing protocols and policies to improve the 423 More research is required on developing protocols and policies to improve the
437 relationship between industry and government, especially regarding CD. 424 relationship between industry and government, especially regarding CD.
438 425
439 ## Acknowledgements 426 ## Acknowledgements
440 427
441 -We thank our colleagues, Nelson Lago, Lucas Kanashiro and Rafael Manzo, and this article's reviewers. 428 +We thank our colleagues, Nelson Lago, Lucas Kanashiro and Rafael Manzo, and
  429 +this article's reviewers.
442 430
443 ## References 431 ## References
444 432
@@ -454,5 +442,3 @@ We thank our colleagues, Nelson Lago, Lucas Kanashiro and Rafael Manzo, and this @@ -454,5 +442,3 @@ We thank our colleagues, Nelson Lago, Lucas Kanashiro and Rafael Manzo, and this
454 9. Chen, L., 2015, May. Towards architecting for continuous delivery. In Software Architecture (WICSA), 2015 12th Working IEEE/IFIP Conference on (pp. 131-134). IEEE. 442 9. Chen, L., 2015, May. Towards architecting for continuous delivery. In Software Architecture (WICSA), 2015 12th Working IEEE/IFIP Conference on (pp. 131-134). IEEE.
455 ---> 443 --->
456 444
457 -  
458 -