Download as
Browse Code »

Commit a15978eb5f8636ae83d4c2ed6092f2adbb15b456

Authored by Paulo Meireles
1 parent 3b83c9d1
Exists in master and in 3 other branches oss_2018, oss_2018_siqueira, rework_case_study

[ieeeSW] General review from Fabio's comments 

Signed-off-by: Paulo Meirelles <paulo@softwarelivre.org>
Signed-off-by: Melissa wen <melissa.srw@gmail.com>
Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
Showing 1 changed file with 225 additions and 274 deletions   Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
  Diff comments   View file @a15978e
1   -<!---
2   -Comment: "Having Gov in the title may turn off some readers. Maybe a title something like "Case study adapting CI to a large-scale, complex government organization" would allow it to cross over to non-gov large orgs."
3   -
4   -Siqueira/Paulo/Fabio: Ainda não chegamos a um título final, mas estamos em vias de convergir
5   --->
6 1 ---
7   -title: "Continuous Delivery: building trust in a large-scale, complex government organization"
  2 +title: "Continuous Delivery: Building Trust in a Large-scale, complex government organization"
8 3 papersize: a4
9 4 geometry: "left=1in,right=1.5in"
10 5 ---
... ... @@ -42,34 +37,27 @@ him at fabio.kon@ime.usp.br.
42 37  
43 38 For many software development teams, the first aspects that come to mind
44 39 regarding Continuous Delivery (CD) are the operational challenges and the
45   -competitive benefits. In our experience, the CD was much more: it was a
46   -survival technique. This article presents how and why we applied CD in a
47   -Brazilian government project for the development of a Collaborative Development
  40 +competitive benefits. In our experience, CD was much more: it was a survival
  41 +technique. This article presents how and why we applied CD in a Brazilian
  42 +government project for the development of a Collaborative Development
48 43 Environment (CDE), sharing the challenges we faced and the strategies used to
49 44 overcome them.
50 45  
51 46 ## Introduction
52   -<!---
53   -Comment: "Generally, the piece comes across as part advertisement for CI and part challenges paper. The authors should figure out their message and make it punch."
54 47  
55   -Siqueira/Paulo: Ao meu ver, a introdução deixa bem claro o nosso "punch" e o resto do texto a gente desenrola bem isso
56   --->
57   -
58   -We worked on a thirty-month-long Brazilian government project to evolve an
59   -existing platform that had technical issues and lacked political support. This
  48 +We worked on a thirty-month-long Brazilian government project to modernize the
  49 +Brazilian Public Software (SPB) portal (www.softwarepublico.gov.br) [1]. This
60 50 project, started in 2014, was a partnership between the Ministry of Planning,
61   -Budget, and Management (MP) and two public universities: University of Brasília
62   -(UnB) and University of São Paulo (USP), to modernize Brazilian Public Software
63   -(SPB) portal (www.softwarepublico.gov.br).
64   -
65   -With this partnership, the SPB portal evolved to a
66   -Collaborative Development Environment (CDE)[1], and this evolution brought
67   -significant benefits not just to the government, but also to society
68   -as a whole. The government could minimize bureaucracy and costs of software
  51 +Budget, and Management and two public universities: University of Brasília and
  52 +University of São Paulo.
  53 +
  54 +With this partnership, the SPB portal evolved to a Collaborative Development
  55 +Environment (CDE) [2] which brought significant benefits for the government and
  56 +the society. The government could minimize bureaucracy and costs of software
69 57 development, encouraging the use of the same set of applications across
70   -different government agencies. The society also gained a mechanism of
71   -transparency and collaboration, since anyone can check the government expenses
72   -on software and contribute to project communities.
  58 +different government agencies. The society gained a mechanism of transparency,
  59 +follow government expenses, and collaboration, contribute to project
  60 +communities. .
73 61  
74 62 In this article, we discuss the use of Continuous Delivery (CD) during our
75 63 experience as the academic partner in this project. We focus on how we managed
... ... @@ -78,24 +66,17 @@ helped to build trust between the government and the university development
78 66 team. CD enabled us to show our progress and earned the government’s confidence
79 67 that we could adequately fulfill their requests, becoming an essential aspect
80 68 of our interaction with them. According to this experience, the use of CD as a
81   -tool to build such trust relationships is yet another of its benefits[2].
  69 +tool to build such trust relationships is yet another of its benefits [3].
82 70  
83 71 ## Context
84   -<!---
85   -Comment: "Background/intro - This could be more streamlined and focused. Maybe center around a question such as - What is different between gov and non-gov context using your data as to illustrate then tie that to what you had to do in your CI process to address this gap."
86   -
87   -Paulo: O ponto não foi gov e não gov. Vamos esclarecer que o SPB foi algo particular mesmo, cheio de nuâncias.
88   -
89   --->
90 72  
91   -The SPB is a governmental program of the MP created to foster sharing and
92   -collaboration on Open Source Software (OSS) development for the Brazilian
93   -public administration. For their projects, the MP managed both software
94   -requirements and server infrastructure. However, its hierarchical and
95   -traditional processes made them unfamiliar with new software development
96   -techniques, such as CD. Any of our requests had to pass through layers of
97   -bureaucracy before being answered. Requesting access to their infrastructure
98   -to make the deploy was not different.
  73 +SPB is a governmental program created to foster sharing and collaboration on
  74 +Open Source Software (OSS) development for the Brazilian public administration.
  75 +For their projects, the Ministry managed both software requirements and server
  76 +infrastructure. However, its hierarchical and traditional processes made them
  77 +unfamiliar with new software development techniques, such as CD. Any of our
  78 +requests had to pass through layers of bureaucracy before being answered,
  79 +accessing their infrastructure to perform a deploy was not different.
99 80  
100 81 During its lifetime, the project suffered significant interference from the
101 82 board of directors because the portal represents an interface between
... ... @@ -106,126 +87,108 @@ directors had different political agendas which affected the project&#39;s
106 87 requirements previously approved.
107 88  
108 89 <!---
109   -Comment:
110   -"The authors present 3 challenges in the intro and then go on to expand them. However, the expansion (line 21-43 in page 2), is has pints about all three challenges mixed together. It would be better I think to split it into one para for each challenge. "
111   -
112   -Siqueira/Paulo: Note que esse ponto gerou muita discussão e confusão. Mudei um pouco a estratégia de forma a responder o revisor e ao mesmo tempo tornar o texto mais fluido
113   -
114   -Melissa: Reformular o enunciado do primeiro problema (FEITO por Siqueira e Diego).
  90 +mudar em todas as ocorrências de agents por staff, mas checar com o Fabio.
115 91 -->
116 92  
117   -In this context, we overcame three distinct challenges: (1) find a system
118   -solution wherein government and development team agree, (2) deconstruct the
119   -widespread belief among the government agents that any project in partnership
120   -with a University is doomed to fail, and (3) deal with bureaucracies involved
121   -in the deployment process by the MP.
122   -
123   -<!---
124   -Comment:
125   -
126   -"Some more details about the project that was developed in terms of what the project did could be shared. Right now it just seems like some platform. I am not sure a platform that does what? And why were these 5 tools (Noosfero, Gitlab, Mailman, Mezuro and Colab) were integrated? What purpose did they each serve?"
127   -
128   -Siqueira/Paulo: Contamos essa história sem ter que entrar em detalhes aqui. Contudo, será preciso fazer a seção de pipeline harmonizar com isso
129   --->
  93 +In this context, we overcame three distinct challenges: (1) finding a system
  94 +solution with which government and development team agree, (2) deconstructing
  95 +the widespread belief among government staff that any project in partnership
  96 +with a University is doomed to fail, and (3) dealing with bureaucracies
  97 +involved in the deployment process by the Ministry.
  98 +
  99 +To face the first issue, we designed the SPB portal as a CDE with additional
  100 +social features. Due to the complexity of creating such a system from scratch,
  101 +we decided to adapt and integrate existing OSS tools to build a
  102 +system-of-systems [4]. We created a solution that orchestrates multiple
  103 +components and allowed us to smoothly provide a unified interface for final
  104 +users, including single sign-on and global searches [1]. On top of that, the
  105 +new SPB portal was an unprecedented platform to the Brazilian government, with
  106 +a complicated deployment process.
  107 +
  108 +
  109 +Regarding the second problem, our team was not from a typical company,
  110 +consisting mainly of undergraduate students coordinated by two professors. In
  111 +the first year, we had a group composed of 24 undergraduate students, one
  112 +designer, and two senior developers. In 2015, our team grew to 36 students, two
  113 +designers, eight senior developers. In the end, due to budget constraints, our
  114 +team shrinked to 20 students, one designer, and two developers. On the
  115 +government side, the SPB portal evolution was the first software development
  116 +collaboration between university and government experienced by the Ministry
  117 +staff involved in the project.
  118 +
  119 +Finally, our team thought software deployment differently than the Ministry. On
  120 +our side, we believe that frequent deliveries are better for the project’s
  121 +success. However, the Ministry works with the idea of a single deployment at
  122 +the end of the project. In other words, neither the bureaucratic structure of
  123 +the Ministry nor its technical abilities were conducive to this style of work.
  124 +Furthermore, there was little effort to deploy new versions of the system
  125 +promptly. That ended up hampering the benefits of the tool and preventing us
  126 +from showing off the fruits of the project to those responsible for evaluating
  127 +it.
  128 +
  129 +These challenges made our relationship with the Ministry staff tense, in
  130 +particular during the first year, and alerted us to the fact that they could
  131 +finish the project at any time. The deployment limitation was the substantial
  132 +technical issue we could tackle in the short term. As a result, we worked to
  133 +deploy one version of the project onto our infrastructure and showed it to the
  134 +government evaluators. This strategy proved them we could efficiently deliver
  135 +new features, fulfill their expectations regarding the delivery of the
  136 +requirements, and incited them to demand that the latest version be deployed in
  137 +the Ministry infrastructure. This generated more pressure on the IT department
  138 +responsible for the deployment routines. With each CD cycle, we gradually built
  139 +a new relationship among all parties and, by the end of the project, we became
  140 +active participants in the deploy operations.
130 141  
131   -To face the first issue, we designed the SPB portal as a CDE with additional social features. Due
132   -to the complexity of creating such a system from scratch, we decided to adapt
133   -and integrate existing OSS tools to build a system-of-systems[1]. We created a
134   -solution that orchestrates software and allowed us to smoothly provide a
135   -unified interface for final users, including single sign-on and global searches.
136   -On top of that, the new SPB portal was an unprecedented platform to the
137   -Brazilian government, with a complicated deployment process.
  142 +## Our Continuous Delivery Pipeline
138 143  
139   -<!---
140   -Comment: "Can the authors provide some data as well? As I said this is a unique scenario and the community can greatly benefit. For example, how many devs at any given time, how many features/unit time (month/year), how frequently were releases made, how frequently did you meet with the client, how frequently did the requirements change are some example questions to which if we had data, it would place the study in greater context."
  144 +![The SPB Deployment Pipeline](figures/pipeline_3.png)
141 145  
142   -Siqueira/Paulo: Mostramos parte dos dados aqui e mais na frente falamos da mudança dupla da diretoria tmb
143   --->
  146 +Figure 1 presents our CD pipeline. It follows a typical deployment pipeline
  147 +[3], adapted to the technical and organizational context of our project and the
  148 +use of OSS best practices. The pipeline started when new code arrived. A new
  149 +feature might require changes to more than one SPB integrated software project.
  150 +Notice that each one of them could be modified independently. As the code went
  151 +through each step, it was tested and improved until it finally reached the
  152 +production environment. At this point, we would restart the pipeline to release
  153 +more versions.
144 154  
145 155 <!---
146   -Comment:
147   -" In page 3, the authors say 10 SW components were integrated, but only 5 were presented in Page 2. I see in Page 3/4 that the authors mean that they used the the tools in page 2 to manage the building of the SW. But that is not clear. This needs to be clarified. "
  156 +Comentário do Fábio: A partir daqui o texto já deveria mostrar o tamanho da plataforma e trazer dados que comprovem isso.
148 157  
149   -Siqueira/Paulo: Isso aqui também fica resolvido
150   --->
151   -
152   -Regarding to the second problem, our team was not from a
153   -typical company, consisting mainly of undergraduate students coordinated by two
154   -professors. At the first year, we had a group composed of 24 undergraduate
155   -students, one designer, and two senior developers. In 2015, our team grew to 36
156   -students, two designers, eight senior developers. In the end, due to budget
157   -constraint, we had 20 students, one designer, and two developers. On the
158   -government side, the SPB portal evolution was the first software development
159   -collaboration between university and government experienced by the MP agents
160   -involved in the project.
161   -
162   -Lastly, our team thought software deployment differently than the MP. On our
163   -side, we believe that frequent deliveries are better for the project’s success.
164   -However, the MP works with the idea of a single deployment at the end of the
165   -project. In other words, neither the bureaucratic structure of the MP nor its
166   -technical abilities were conducive to this style of work. Furthermore, there
167   -was little effort to deploy new versions of the system promptly. That ended up
168   -hampering the benefits of the tool and preventing us from showing off the
169   -fruits of the project to those responsible for evaluating it.
  158 +https://www.openhub.net/p/gitlab
  159 +https://www.openhub.net/p/noosfero
  160 +https://www.openhub.net/p/mezuro
  161 +https://www.openhub.net/p/mailman
170 162  
171   -<!---
172   -"the article is missing the ah,ha moment. Was there something interesting that you learned (could be a small thing) that you use to adapt the CI process when applying it in a gov or large org context versus a smaller org? "
173 163  
174   -Siqueira/Paulo: eis o AH-HA moment!
175 164 -->
176 165  
177   -These challenges made our relationship with the MP agents tense, in particular
178   -at the first year, and alerted us to the fact that they could finish the
179   -project at any time. The deployment limitation was the substantial technical
180   -issue we could tackle in the short term. As a result, we worked to deploy one
181   -version of the project onto our infrastructure and showed it to the government
182   -evaluators. This strategy proved them we could efficiently deliver new
183   -features, fulfill their expectations regarding the delivery of the
184   -requirements, and incited them to demand the last version working in the MP
185   -infrastructure. These results, in turn, generated more pressure on the IT
186   -department responsible for the deployment routines. With each CD cycle, we
187   -gradually built a new relationship among all parties and, by the end of the
188   -project, we became active participants in the deploy operations.
189   -
190   -## Our Continuous Delivery Pipeline
191   -
192   -![Deployment Pipeline](figures/pipeline_3.png)
193   -
194   -Figure 1 represents our CD pipeline. It follows a typical deployment
195   -pipeline[5], adapted to the technical and organizational context of our project
196   -and the use of OSS projects practices. The pipeline started when a code arrived.
197   -A new feature might require changes to more than one SPB integrated software
198   -project. Notice that each one of them could be modified independently. As the
199   -code went through each step, it was tested and improved until it finally reached
200   -the production environment. At this point, we would restart the pipeline to
201   -release more versions.
202   -
203 166 ### Automated Tests
204 167  
205 168 <!---
206   -Diego: Deixei a mini explicação usando o Colab porque achei que ainda faz sentido
207   -e está bem colocada. Se decidirmos tirar, teremos que repensar o parágrafo.
  169 +No final do contexto, adicionar um parágrafo final falando que geramos o SPB de forma a integrar 5 sistemas e manter mais de 30 pacotes.
  170 +Os 5 projetos são: Colab, Noosfero, Gitlab, MailMan, and Mezuro.
208 171 -->
209 172  
210   -The SPB portal is a system-of-systems with several integrated software
211   -projects. Each of them, as well as the entire platform, had to be tested.
212   -These software components have own test suite. Colab (www.github.com/colab), a
213   -systems integration platform for web applications based on a plugins
214   -architecture, orchestrate communication between all of them. Therefore, we
215   -developed specific plugins for each portal software component, such as Gitlab
216   -(www.gitlab.com) and Noosfero (www.noosfero.org). Each plugin also has own test
217   -suite, and this set also worked as integration tests.
  173 +The SPB portal is a system-of-systems with 5 integrated software projects. Each
  174 +of them, as well as the entire platform, had to be tested. These software
  175 +components have their own test suite. Colab (www.github.com/colab), a systems
  176 +integration platform for web applications based on a plugin architecture,
  177 +orchestrates communication among them. Therefore, we developed specific plugins
  178 +for each portal software component, such as Gitlab (www.gitlab.com) and
  179 +Noosfero (www.noosfero.org). Each plugin also has its own test suite, working
  180 +also as integration tests.
218 181  
219 182 Both unit and integration tests provided us the performance and security needed
220 183 to guarantee the stability of components and the platform. If any test suite
221 184 failed, by either a test error or coverage reduction below a certain threshold,
222 185 the process stopped. Only when all tests passed, the pipeline proceeded to the
223   -step of preparing the release.
  186 +step of release preparation.
224 187  
225 188 ### Preparing a New Release
226 189  
227   -An SPB portal release was composed of all its software components releases.
228   -Each software component release was a Git tag that referred to a specific
  190 +An SPB portal release was composed of all its software component releases.
  191 +Each software component release had a Git tag that referred to a specific
229 192 feature or bug fix. When all tests passed for a given component, we manually
230 193 created a new tag for it. Therefore, a new tag on any software component
231 194 yielded a new SPB portal release. More precisely, SPB had a script that
... ... @@ -234,206 +197,194 @@ the end of this process, we started packaging.
234 197  
235 198 ### Packaging
236 199  
237   -The platform is running on the CentOS 7 GNU/Linux distribution. Packaging a
238   -software for that distribution has three steps: write the script for the
239   -specific environment (RPM), build the package, and upload it to a package
  200 +The platform runs on the CentOS 7 GNU/Linux distribution. Packaging a software
  201 +for that distribution involves three steps: writing the script for the specific
  202 +environment (RPM), building the package, and uploading it to a package
240 203 repository.
241 204  
242   -We decided to create own packages for each software component for the following
243   -five reasons:
244   -
245   -1. The community packaged not all software, and those that existed were outdated;
246   -1. Packaging makes it easy to manage the software on a given distribution;
247   -1. Packaging simplifies the deployment;
248   -1. Packaging follows the distribution's best practices and,
249   -1. Packaging allows configurations and permissions control.
  205 +We decided to create separate packages for each software component since:
  206 +Packaging makes easy to manage the software on a given distribution,
  207 +simplifies the deployment, follows the distribution's best practices, and
  208 +enables configurations and permissions control.
250 209  
251   -After creating a new tag for one component, the developers informed our DevOps
  210 +After creating a new tag for a component, the developers informed our DevOps
252 211 [6] team, and the packaging process began. A set of scripts fully automated the
253   -three packaging steps aforementioned. With all them running successfully, the
254   -new packages would be ready to be used by our deployment scripts.
  212 +three packaging steps aforementioned. When all them ran successfully, the new
  213 +packages would be ready for our deployment scripts.
255 214  
256 215 ### Validation Environment Deployment
257 216  
258   -The Validation Environment (VE) is a replica of the Production Environment
259   -(PE), with two exceptions: only the government officers and project leaders had
260   -access to it and all the data became anonymous. To configure the environment,
261   -we used a configuration management tool named Chef with Chake support
262   -(www.github.com/terceiro/chake) -- a serverless configuration created by our
263   -team). It maintained environment consistency simplifying the deployment
  217 +The Validation Environment (VE) is a replica of the Production Environment (PE)
  218 +with its data anonymised, as well as only Ministry staffs and our DevOps team
  219 +had access to it. To configure the environment, we used a configuration
  220 +management tool named Chef (www.chef.io) with Chake support
  221 +(www.github.com/terceiro/chake) -- a serverless configuration tool created by
  222 +our team. It maintained environment consistency simplifying the deployment
264 223 process. Additionally, the packages we built on the last step were readily
265   -available to be used by the management tool.
  224 +available to the management tool.
266 225  
267   -The MP agents used the VE to validate new features and required changes. The VE
268   -also was used to verify the integrity of the entire portal as part of the next
269   -step in the pipeline.
  226 +The Ministry staff used the VE to validate new features and required changes.
  227 +The VE also was used to verify the integrity of the entire portal as part of
  228 +the next step in the pipeline.
270 229  
271 230 ### Acceptance Tests
272 231  
273   -After we deployed a new SPB portal version in the VE, the MP agents were
274   -responsible for checking features and bug fixes required by them. If the
275   -MP agents identified a problem, they would notify the developers via
  232 +After we deployed a new SPB portal version in the VE, the Ministry staffs were
  233 +responsible for checking the features and bug fixes they required. If the
  234 +Ministry staffs identified a problem, they would notify the developers via
276 235 comments on the SPB portal's issue tracker. The development team fixed the
277   -problem and the pipeline restarted from scratch. If everything was validated,
278   -we moved forward.
  236 +problem and the pipeline restarted. If everything was validated, we moved
  237 +forward.
279 238  
280 239 ### Production Environment Deployment
281 240  
282   -When the MP agents finished the VE check, we could finally begin the deployment
283   -in the PE. For this, we also used our configuration management tool, the same
284   -scripts and package versions as in the VE. After the deploy was completed, both
285   -VE and PE were running identical software. Here was the point where new
286   -features and bug fixes were finally available to the end users.
  241 +When the Ministry staff finished the VE check, we could finally begin the
  242 +deployment in production. We also used our configuration management tool, the
  243 +same scripts and package versions as in the VE. After the deploy was completed,
  244 +both VE and PE were identical. Here was the point where new features and bug
  245 +fixes were finally available to end users.
287 246  
288 247 ## Benefits
289 248  
290   -Research points out many advantages of CD usage in the industry[2, 7], such as
  249 +Research points out many CD advantages in the industry [2, 7], such as
291 250 accelerated time to market, building the right product, productivity and
292 251 efficiency improvements, stable releases, and better customer satisfaction.
293 252 Working with the government, we noticed the following additional benefits.
294 253  
295   -### Strengthening Trust in Work Relationship with the Government
  254 +### Strengthening Trust in the Relationship with the Government
296 255  
297   -Continuous delivery was also a tool that helped to strengthen trust in the
298   -relationship between developers and MP agents. Before using CD, the MP agents
299   -had access to the features developed only at the end of the release, usually
300   -every four months.
  256 +CD helped to strengthen trust in the relationship between developers and
  257 +Ministry staffs. Before using CD, Ministry staff had access to the features
  258 +developed only at the end of the release, usually every four months.
301 259  
302 260 With the implementation of CD, intermediate and candidate versions became
303   -available, allowing the MP agents to perform small validations over time. The
304   -constant monitoring of the development work brought greater security to the MP
305   -leaders and improved the interactions with our development team.
  261 +available, allowing Ministry staffs to perform small validations over time.
  262 +Constant monitoring of the development work brought greater security to the
  263 +Ministry leaders and improved the interactions with our development team.
306 264  
307 265 ### Responsiveness to Change
308 266  
309 267 Responsiveness was one of the direct benefits of adopting the CD pipeline. The
310   -ability to react quickly to changes requested by the MP agents was vital for
311   -the renewal of the project over the years. Every meeting with the MP leaders
312   -resulted in requirements and priorities changes, several of them motivated by
313   -political needs. We observed that if we took too long to attend their demands,
314   -the MP would use undelivered requirements as a means to justify the lack of
315   -financial support and the end of the project.
316   -
317   -CD helped us keep the production environment up-to-date, even with partial
318   -versions of a feature. That way, we always had something to show on meetings,
319   -reducing anxiety to get the platform concluded. For our team, it made the
320   -developers more confident that the project would last a little longer and they
321   -would not go looking for other jobs.
  268 +ability to react quickly to changes requested by the Ministry staff was vital
  269 +to the project’s survival for 30 months. Every meeting with the Ministry
  270 +leaders resulted in requirements and priorities changes, several of them
  271 +motivated by political needs. We observed that if we took too long to meet
  272 +their demands, the Ministry would use undelivered requirements to justify cut
  273 +in the financial support and cancel the project.
  274 +
  275 +CD helped us keep the PE up-to-date, even with partial versions of a feature.
  276 +That way, we always had something to show on meetings, reducing anxiety in
  277 +getting the platform finished. For our team, it made the developers more
  278 +confident that the project would last a little longer.
322 279  
323 280 ### Shared Responsibility
324 281  
325   -According to the MP process, the development team could not track what happened
326   -to the code after its delivery, since the MP agents were the only ones
327   -responsible for deployment. The implementation of CD made our development team
328   -feel equally responsible for what was getting into production and take
329   -ownership of the project.
  282 +According to the conventional Ministry process, the development team could not
  283 +track what happened to the code after its delivery, since Ministry staff were
  284 +the only ones responsible for deployment. The implementation of CD made our
  285 +development team feel equally responsible for what was getting into production
  286 +and take ownership of the project.
330 287  
331   -Interestingly, the CD pipeline had the same effect on the MP agents. They
  288 +Interestingly, the CD pipeline had the same effect on the Ministry staff. They
332 289 became more engaged in the whole process, opening and discussing issues during
333   -the platform evolution. Additionally, developers worked to improve the CD
334   -pipeline to speed up the process of making new features available in the
335   -production environment for the MP agents' validation.
  290 +platform evolution. Additionally, developers worked to improve the CD pipeline
  291 +to speed up the process of making new features available in the production
  292 +environment for the Ministry staff's validation.
336 293  
337 294  
338 295 ### Synchronicity Between Government and Development
339 296  
340   -Despite the positive impacts that the CD pipeline brought to the project, its
341   -implementation was not smooth at first. The CD pipeline performance depended on
342   -the synchronicity between our development team and the MP agents so that the
343   -latter were prepared to start a step as soon as the former concluded the
344   -previous step and vice versa. Initially, the agenda of the MP agents did not
345   -contemplate this concern, which generated delays in the validation of new
346   -features. This situation combined with governmental bureaucracy (up to 3 days)
347   -to release access to the production environment resulted in additional delays
348   -for the deployment step to begin. This problem was softened when the MP
349   -agents realized the impact of these delays on the final product and decided
350   -to allocate the revisions in its work schedule and to request the access to
351   -production in time.
352   -
353   -## Challenges
354   -
355   -Due to the successful building of the CD pipeline, we improved the MP
356   -deployment process and kept the project alive. We map here lessons learned in
357   -this successful case.
  297 +The CD pipeline performance depended on the synchronicity between our
  298 +development team and the Ministry staffs so that the latter were prepared to
  299 +start a step as soon as the former concluded the previous step and vice versa.
  300 +Initially, the agenda of the Ministry staffs did not contemplate this concern,
  301 +which generated delays in the validation of new features. This situation
  302 +combined with governmental bureaucracy to release access to the production
  303 +environment (up to 3 days) resulted in additional delays for the deployment
  304 +step begin. This problem was softened when the Ministry staff realized the
  305 +impact of these delays on the final product and decided to allocate the
  306 +revisions in their work schedule.
  307 +
  308 +<!---
  309 +Fabio sugeriu Lessons Learned, mas vamos mostrar exemplos da revista para ele olhar.
  310 +-->
  311 +
  312 +## Lessons Learned
  313 +
  314 +Due to the successful building of the CD pipeline, we improved the Ministry
  315 +deployment process and kept the project alive. We map now lessons learned.
358 316  
359 317 ### Build CD From Scratch
360 318  
361 319 Taking on responsibilities for implementing CD impacted on the whole team.
362 320 Mostly, our team members did not have know-how in this approach, and we had few
363   -working hours available to allocate for the building of the pipeline. The
364   -construction and maintenance of the CD process were possible by taking some
365   -decisions to mature the project:
  321 +working hours available to allocate for building the pipeline. The construction
  322 +and maintenance of the CD process were possible by taking some decisions to
  323 +mature the project:
366 324  
367   -1. _Select the most experienced senior developers and some advanced software
368   -engineering students of the project to work on a specific team for DevOps._
369   -These senior developers used their experiences in OSS projects to get an
370   -initial proposal for the deployment process. The solution enabled us to
371   -automate the deployment, even though the process was still rudimentary.
  325 +<!---
  326 +pensar em generalizar/filosofar
  327 +-->
  328 +
  329 +1. _Select the most experienced senior developers and some advanced students of
  330 +the project to work on a specific DevOps team._These senior developers used
  331 +their experiences in OSS projects to craft an initial proposal for the
  332 +deployment process. The solution enabled us to automate the deployment, even
  333 +though the process was, initially, still rudimentary.
372 334  
373   -2. _Interchange team members and encouraging teammates to migrate to DevOps
  335 +2. _Interchange team members and encourage teammates to migrate to the DevOps
374 336 team._ The benefits of these movements were twofold: mitigating the difficulty
375   -to pass the knowledge between DevOps developers and features developers, and
  337 +to transmit the knowledge between DevOps developers and feature developers, and
376 338 evolving the process on-the-fly.
377 339  
378   -Building a CD pipeline was hard in the beginning. We believe that more tools to
379   -provide out-of-the-box standardized CD pipelines would be of great help for
380   -inexperienced teams. Tools that track each step of the pipeline and organize
381   -logs in a human-manageable way are necessary too.
382   -
383 340 ### Overcoming Mistrust
384 341  
385   -Taking an unfamiliar approach requires trust. In the MP, software is the
386   -product delivered at the end of a development contract. They expected and were
387   -prepared to validate and deploy a single delivery. Because the SPB portal is a
388   -system-of-systems, the steady growth of its complexity made large deliveries
389   -unsustainable. The long time for homologation of developed features also gave
390   -the government room to change requirements and priorities. The CD approach was
391   -necessary, but how to build trust and gain autonomy to implement a process that
392   -was not yet part of the dynamics of the Ministry?
  342 +Taking an unfamiliar approach requires trust. In the Ministry, traditional
  343 +software was the product delivered at the end of a development contract. They
  344 +expected and were prepared to validate and deploy a single delivery. Because
  345 +the SPB portal is a system-of-systems, the steady growth of its complexity made
  346 +large deliveries unsustainable. The long time for homologation of developed
  347 +features also gave the government room to change requirements and priorities.
  348 +The CD approach was necessary, but how to build trust and gain autonomy to
  349 +implement a process that was not yet part of the dynamics of the Ministry?
393 350  
394 351 1. _Demonstrate actual results, do not simply tell._ Initially, we did not have
395   -access to the MP infrastructure, so we created our own validation environment.
396   -Thus, we were able to follow the CD pipeline until the stage of production
397   -deployment, when we faced two problems. Our pace of intermediate deliveries to
398   -the government was faster than the deployment in production by the MP
399   -agents. Furthermore, specific issues of the MP infrastructure made some
400   -validated features not work as expected in the PE. That situation gave us
401   -arguments to negotiate access to PE.
402   -
403   -2. _Make our project management transparent and collaborative for the MP
404   -agents._ Allowing the MP agents to follow our process for version deliveries
405   -and bug fixes, we showed them we were meeting our commitments. They started to
406   -interact more actively in the generation of versions and became part of the
407   -process. After understanding the process, the MP agents helped us in
408   -negotiations with the MP leaders. Finally, they created a VE as
409   -an isolated replica of PE and gave us access to it.
410   -
411   -3. _Gain the confidence of government agents._ With the replica of PE, we were
412   -able to run the entire pipeline and won the trust of the MP agents involved in
413   -the process. They saw the mobilization and responsiveness of our team to
414   -generate a new version package. They also recognized the quality of our
415   -packages and our deployment process. Finally, the MP agents then realized that
416   -it could be beneficial for the project if they granted us access to the project
  352 +access to the Ministry infrastructure, so we created our own validation
  353 +environment. Thus, we were able to follow the CD pipeline until the stage of
  354 +production deployment, when we faced two problems. First, our pace of
  355 +intermediate deliveries to the government was faster than the deployment in
  356 +production by the Ministry staff. Second, specific issues of the Ministry
  357 +infrastructure made some validated features not work as expected in the PE.
  358 +That situation gave us arguments to negotiate access to production.
  359 +
  360 +2. _Make project management transparent and collaborative for government
  361 +staff._ Allowing the Ministry staff to follow our process for version
  362 +deliveries and bug fixes, we showed them we were fulfilling our commitments.
  363 +They started to interact more actively in the generation of versions and became
  364 +part of the process. After understanding the process, the Ministry staff helped
  365 +us in negotiations with the Ministry leaders. Finally, they created a VE as an
  366 +isolated replica of the PE and gave us access to it.
  367 +
  368 +3. _Gain the confidence of government staff._ With the replica of the PE, we
  369 +were able to run the entire pipeline and won the trust of the Ministry staff
  370 +involved in the process. They saw the mobilization and responsiveness of our
  371 +team to generate a new version package. They also recognized the quality of our
  372 +packages and our deployment process. Finally, the Ministry staff then realized
  373 +that it could be beneficial for the project if they granted us access to the
417 374 infrastructure, both VE and PE.
418 375  
419 376 <!---
420   -Paulo: Acho que precisamos de algo ligado ao Ha-Ha-moment para fechar o texto aqui.
  377 +Paulo: Acho que precisamos de algo ligado ao Ha-Ha-moment para fechar o texto aqui; ou fechar falando que tudo foi feito de forma aberta e colaborativa com as comunidades dos projetos envolvidos, estão todos os fontes disponíveis em https://softwarepublico.gov.br/gitlab/softwarepublico/
421 378 -->
422 379  
423 380 ## References
424 381  
425   -1. P .Meirelles, M. Wen, A. Terceiro, R. Siqueira, L. Kanashiro, and H. Neri, "Brazilian Public Software Portal: an integrated platform for collaborative development", Proceedings of the 13th International Symposium on Open Collaboration. ACM, Article 16, 2017, 10 pages.
  382 +1. P. Meirelles, M. Wen, A. Terceiro, R. Siqueira, L. Kanashiro, and H. Neri, "Brazilian Public Software Portal: an integrated platform for collaborative development", Proceedings of the 13th International Symposium on Open Collaboration. ACM, Article 16, 2017, 10 pages.
  383 +1. G. Booch and A. Brown, A. W. "Collaborative Development Environments", Advances in Computers, vol. 59, 2003, pp. 1-27.
426 384 1. L. Chen, "Continuous Delivery: Huge Benefits, but Challenges Too", IEEE Software, vol. 32, no. 2, 2015, pp. 50-54.
  385 +1. C. B. Nielsen, P. G. Larsen, J. Fitzgerald, J. Woodcock, and J. Peleska, "Systems of Systems Engineering: Basic Concepts, Model-Based Techniques, and Research Directions", ACM Comput. Surv. 48, 2, Article 18, 2015, 41 pages.
427 386 1. J. Humble and D. Farley, "Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation", Addison-Wesley Professional, 2010.
428 387 1. J. Davis and K. Daniels, "Effective Devops: Building a Culture of Collaboration, Affinity, and Tooling at Scale", O'Reilly Media, Inc., 2016.
429 388 1. T. Savor, M. Douglas, M. Gentili, L. Williams, K. Beck and M. Stumm, "Continuous Deployment at Facebook and OANDA", 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C), Austin, TX, 2016, pp. 21-30.
430 389  
431 390  
432   -<!---
433   -Paulo: Entendo que neste tipo de texto não precisamos de referências para conceituar CDE e SoS
434   -
435   -1. G. Booch and A. Brown, A. W. "Collaborative Development Environments", Advances in Computers, vol. 59, 2003, pp. 1-27.
436   -1. C. B. Nielsen, P. G. Larsen, J. Fitzgerald, J. Woodcock, and J. Peleska, "Systems of Systems Engineering: Basic Concepts, Model-Based Techniques, and Research Directions", ACM Comput. Surv. 48, 2, Article 18, 2015, 41 pages.
437   -
438   --->
439   -
... ...