Commit 112263f9ac061e228ad6cb070e2e078dd8681949

Authored by Melissa Wen
1 parent 52d15058

[i3eSW] Remove untracked references

ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
... ... @@ -11,19 +11,42 @@ geometry: "left=1in,right=1.5in"
11 11  
12 12 ## Authors
13 13  
14   -__Rodrigo Siqueira__ is a masters student at IME - The Institute of Mathematics and Statistics of the Sao Paulo University. His research interests include software engineering, operating system and computer architecture. He worked for two years with the Brazilian federal government as a coach and developer. Contact him at siqueira@ime.usp.br.
15   -
16   -__Diego Camarinha__ is a masters student at IME - The Institute of Mathematics and Statistics of the Sao Paulo University. His research interests include software engineering, computer networks and source code metrics. He worked for two years with the Brazilian federal government as a senior developer. Contact him at diegoamc@ime.usp.br.
17   -
18   -__Melissa Wen__ is a software developer. She worked on SPB project as a senior developer and also served as professor of Computer Science at UFBA - The Federal University of Bahia. Her areas of interest include software engineering and open source software development. Contact her at melissa.srw@gmail.com .
19   -
20   -__Paulo Meirelles__ received his Ph.D. in Computer Science from the Institute of Mathematics and Statistics at the University of São Paulo. He is full-time Professor at the University of Brasilia, and coordinated the new SPB Portal project. His research interest areas are: Free Software, Agile methods, Static analysis, and Source code metrics. Contact him at paulormm@ime.usp.br.
21   -
22   -__Fabio Kon__ is Full Professor of the Department of Computer Science of the University of São Paulo. (...) .His research interest areas are: (...). Contact him at fabio.kon@ime.usp.br.
  14 +__Rodrigo Siqueira__ is a masters student at IME - The Institute of Mathematics
  15 +and Statistics of the Sao Paulo University. His research interests include
  16 +software engineering, operating system and computer architecture. He worked for
  17 +two years with the Brazilian federal government as a coach and developer.
  18 +Contact him at siqueira@ime.usp.br.
  19 +
  20 +__Diego Camarinha__ is a masters student at IME - The Institute of Mathematics
  21 +and Statistics of the Sao Paulo University. His research interests include
  22 +software engineering, computer networks and source code metrics. He worked for
  23 +two years with the Brazilian federal government as a senior developer. Contact
  24 +him at diegoamc@ime.usp.br.
  25 +
  26 +__Melissa Wen__ is a software developer. She worked on SPB project as a senior
  27 +developer and also served as professor of Computer Science at UFBA - The Federal
  28 +University of Bahia. Her areas of interest include software engineering and open
  29 +source software development. Contact her at melissa.srw@gmail.com .
  30 +
  31 +__Paulo Meirelles__ received his Ph.D. in Computer Science from the Institute of
  32 +Mathematics and Statistics at the University of São Paulo. He is full-time
  33 +Professor at the University of Brasilia, and coordinated the new SPB Portal
  34 +project. His research interest areas are: Free Software, Agile methods, Static
  35 +analysis, and Source code metrics. Contact him at paulormm@ime.usp.br.
  36 +
  37 +__Fabio Kon__ is Full Professor of the Department of Computer Science of the
  38 +University of São Paulo. (...) .His research interest areas are: (...). Contact
  39 +him at fabio.kon@ime.usp.br.
23 40  
24 41 ## Abstract
25 42  
26   -For many software development teams, the first aspects that come to mind regarding Continuous Delivery (CD) are the operational challenges and the competitive benefits. In our experience, the CD was much more: it was a survival technique. This article presents how and why we applied CD in a Brazilian government project for the development of a Collaborative Development Environment (CDE), sharing the challenges we faced and the strategies used to overcome them.
  43 +For many software development teams, the first aspects that come to mind
  44 +regarding Continuous Delivery (CD) are the operational challenges and the
  45 +competitive benefits. In our experience, the CD was much more: it was a survival
  46 +technique. This article presents how and why we applied CD in a Brazilian
  47 +government project for the development of a Collaborative Development
  48 +Environment (CDE), sharing the challenges we faced and the strategies used to
  49 +overcome them.
27 50  
28 51 ## Introduction
29 52 <!---
... ... @@ -32,11 +55,30 @@ Comment: &quot;Generally, the piece comes across as part advertisement for CI and par
32 55 Siqueira/Paulo: Ao meu ver, a introdução deixa bem claro o nosso "punch" e o resto do texto a gente desenrola bem isso
33 56 -->
34 57  
35   -We worked on a three-year-long Brazilian government project to evolve an existing platform that had technical issues and lacked political support. This project, started in 2014, was a partnership between the Ministry of Planning, Budget, and Management (MP) and two public universities: University of Brasília (UnB) and University of São Paulo (USP), to modernize Brazilian Public Software (SPB) portal.
36   -
37   -With this partnership, the SPB Portal (www.softwarepublico.gov.br) evolved to a Collaborative Development Environment[1], and this evolution brought significant benefits not just to the Brazilian government, but also to society as a whole. The government could minimize bureaucracy and costs of software development, encouraging the use of the same set of applications across different government agencies. The society also gained a mechanism of transparency and collaboration, since anyone can check the government expenses on software and contribute to project communities.
38   -
39   -In this article, we discuss the use of Continuous Delivery (CD) during our experience as the academic partner in this project. We focus on how we managed to implement CD in a large institution with traditional values and how CD helped to build trust between the government and the university development team. CD enabled us to show our progress and earned the government’s confidence that we could adequately fulfill their requests, becoming an essential aspect of our interaction with them. According to this experience, the use of CD as a tool to build such trust relationships is yet another of its benefits[2].
  58 +We worked on a three-year-long Brazilian government project to evolve an
  59 +existing platform that had technical issues and lacked political support.
  60 +This project, started in 2014, was a partnership between the Ministry of
  61 +Planning, Budget, and Management (MP) and two public universities:
  62 +University of Brasília (UnB) and University of São Paulo (USP), to
  63 +modernize Brazilian Public Software (SPB) portal.
  64 +
  65 +With this partnership, the SPB Portal (www.softwarepublico.gov.br) evolved to a
  66 +Collaborative Development Environment[1], and this evolution brought significant
  67 +benefits not just to the Brazilian government, but also to society as a whole.
  68 +The government could minimize bureaucracy and costs of software development,
  69 +encouraging the use of the same set of applications across different government
  70 +agencies. The society also gained a mechanism of transparency and collaboration,
  71 +since anyone can check the government expenses on software and contribute to
  72 +project communities.
  73 +
  74 +In this article, we discuss the use of Continuous Delivery (CD) during our
  75 +experience as the academic partner in this project. We focus on how we managed
  76 +to implement CD in a large institution with traditional values and how CD helped
  77 +to build trust between the government and the university development team. CD
  78 +enabled us to show our progress and earned the government’s confidence that we
  79 +could adequately fulfill their requests, becoming an essential aspect of our
  80 +interaction with them. According to this experience, the use of CD as a tool to
  81 +build such trust relationships is yet another of its benefits[2].
40 82  
41 83 ## Context
42 84 <!---
... ... @@ -46,9 +88,21 @@ Paulo: O ponto não foi gov e não gov. Vamos esclarecer que o SPB foi algo part
46 88  
47 89 -->
48 90  
49   -The SPB is a governmental program of the MP created to foster sharing and collaboration on Open Source Software (OSS) development for the Brazilian public administration. For their projects, the MP managed both software requirements and server infrastructure. However, its hierarchical and traditional processes made them unfamiliar with new software development techniques, such as CD. Any of our requests had to pass through layers of bureaucracy before being answered. Requesting access to their infrastructure to make the deploy was not different.
50   -
51   -During its lifetime, the project suffered significant interference from the board of directors because the portal represents an interface between government and society. In light of political interests, directors continually imposed changes to the platform while ignoring our technical advice. In 2015, the board of directors was changed and, with it, the vision of the project. New directors had different political agendas which affected the project's requirements previously approved.
  91 +The SPB is a governmental program of the MP created to foster sharing and
  92 +collaboration on Open Source Software (OSS) development for the Brazilian public
  93 +administration. For their projects, the MP managed both software requirements
  94 +and server infrastructure. However, its hierarchical and traditional processes
  95 +made them unfamiliar with new software development techniques, such as CD. Any
  96 +of our requests had to pass through layers of bureaucracy before being answered.
  97 +Requesting access to their infrastructure to make the deploy was not different.
  98 +
  99 +During its lifetime, the project suffered significant interference from the
  100 +board of directors because the portal represents an interface between government
  101 +and society. In light of political interests, directors continually imposed
  102 +changes to the platform while ignoring our technical advice. In 2015, the board
  103 +of directors was changed and, with it, the vision of the project. New directors
  104 +had different political agendas which affected the project's requirements
  105 +previously approved.
52 106  
53 107 <!---
54 108 Comment:
... ... @@ -59,7 +113,11 @@ Siqueira/Paulo: Note que esse ponto gerou muita discussão e confusão. Mudei um
59 113 Melissa: Reformular o enunciado do primeiro problema (FEITO por Siqueira e Diego).
60 114 -->
61 115  
62   -In this context, we overcame three distinct challenges: (1) find a system solution wherein government and development team agree, (2) deconstruct the widespread belief among the government agents that any project in partnership with a University is doomed to fail, and (3) deal with bureaucracies involved in the deployment process by the MP.
  116 +In this context, we overcame three distinct challenges: (1) find a system
  117 +solution wherein government and development team agree, (2) deconstruct the
  118 +widespread belief among the government agents that any project in partnership
  119 +with a University is doomed to fail, and (3) deal with bureaucracies involved in
  120 +the deployment process by the MP.
63 121 <!---
64 122 Comment:
65 123  
... ... @@ -68,7 +126,14 @@ Comment:
68 126 Siqueira/Paulo: Contamos essa história sem ter que entrar em detalhes aqui. Contudo, será preciso fazer a seção de pipeline harmonizar com isso
69 127 -->
70 128  
71   -Firstly, to find a system solution wherein government and development team agree, we designed the SPB Portal as a CDE with additional social features. Due to the complexity of creating such a system from scratch, we decided to adapt and integrate existing open source tools to build a system-of-systems [3]. We created a solution that orchestrates software and allowed us to smoothly provide a unified interface for final users, including single sign-on and global searches [4]. On top of that, the new SPB Portal was an unprecedented platform to the Brazilian government, with a complicated deployment process.
  129 +Firstly, to find a system solution wherein government and development team
  130 +agree, we designed the SPB Portal as a CDE with additional social features. Due
  131 +to the complexity of creating such a system from scratch, we decided to adapt
  132 +and integrate existing open source tools to build a system-of-systems [3]. We
  133 +created a solution that orchestrates software and allowed us to smoothly provide
  134 +a unified interface for final users, including single sign-on and global
  135 +searches [4]. On top of that, the new SPB Portal was an unprecedented platform
  136 +to the Brazilian government, with a complicated deployment process.
72 137  
73 138 <!---
74 139 Comment: "Can the authors provide some data as well? As I said this is a unique scenario and the community can greatly benefit. For example, how many devs at any given time, how many features/unit time (month/year), how frequently were releases made, how frequently did you meet with the client, how frequently did the requirements change are some example questions to which if we had data, it would place the study in greater context."
... ... @@ -95,13 +160,29 @@ Paulo: o segundo problema foi alterado em relação à primeira versão, mas a e
95 160 Siqueira: trocar conhecimento técnico por interesse, disponibilidade ou esforço ...
96 161 -->
97 162  
98   -Secondly, we had to face the widespread belief among government agents that any project in partnership with a University is doomed to fail. Our team was not from a typical company, consisting mainly of undergraduate students coordinated by two professors. At the first year, we had a group composed of 24 undergraduate students, one designer, and two senior developers. In 2015, our team grew to 36 students, two designers, eight senior developers. At the end of in project, due to budget constraint, we had 20 students, one designer, and two developers. Also, on the government side, the SPB Portal evolution was the first software development collaboration between university and government experienced by the MP analysts involved in the project.
  163 +Secondly, we had to face the widespread belief among government agents that any
  164 +project in partnership with a University is doomed to fail. Our team was not
  165 +from a typical company, consisting mainly of undergraduate students coordinated
  166 +by two professors. At the first year, we had a group composed of 24
  167 +undergraduate students, one designer, and two senior developers. In 2015, our
  168 +team grew to 36 students, two designers, eight senior developers. At the end of
  169 +in project, due to budget constraint, we had 20 students, one designer, and two
  170 +developers. Also, on the government side, the SPB Portal evolution was the first
  171 +software development collaboration between university and government experienced
  172 +by the MP analysts involved in the project.
99 173  
100 174 <!---
101 175 Paulo: o terceiro problema é novo em relação à primeira versão, mas na prática tínhamos tratado a questão
102 176 -->
103 177  
104   -Lastly, our team thought software deployment differently than the MP. On our side, we believe that frequent deliveries are better for the project’s success. However, the MP works with the idea of a single deployment at the end of the project. In other words, neither the bureaucratic structure of the MP nor its technical abilities were conducive to this style of work. Furthermore, there was little effort to deploy new versions of the system promptly. That ended up hampering the benefits of the tool and preventing us from showing off the fruits of the project to those responsible for evaluating it.
  178 +Lastly, our team thought software deployment differently than the MP. On our
  179 +side, we believe that frequent deliveries are better for the project’s success.
  180 +However, the MP works with the idea of a single deployment at the end of the
  181 +project. In other words, neither the bureaucratic structure of the MP nor its
  182 +technical abilities were conducive to this style of work. Furthermore, there was
  183 +little effort to deploy new versions of the system promptly. That ended up
  184 +hampering the benefits of the tool and preventing us from showing off the fruits
  185 +of the project to those responsible for evaluating it.
105 186  
106 187 <!---
107 188 "the article is missing the ah,ha moment. Was there something interesting that you learned (could be a small thing) that you use to adapt the CI process when applying it in a gov or large org context versus a smaller org? "
... ... @@ -109,7 +190,18 @@ Lastly, our team thought software deployment differently than the MP. On our sid
109 190 Siqueira/Paulo: eis o AH-HA moment!
110 191 -->
111 192  
112   -These challenges made our relationship with the MP tense, in particular at the first year, and alerted us to the fact that they could finish the project at any time. The deployment limitation was the substantial technical issue we could tackle in the short term. As a result, we worked to deploy one version of the project onto our infrastructure and showed it to the government evaluators. This strategy proved them we could efficiently deliver new features, fulfill their expectations regarding the delivery of the requirements, and incited them to demand the last version working in the MP infrastructure. These results, in turn, generated more pressure on the IT department responsible for the deployment routines. With each CD cycle, we gradually built a new relationship among all parties and, by the end of the project, we became active participants in the deploy operations.
  193 +These challenges made our relationship with the MP tense, in particular at the
  194 +first year, and alerted us to the fact that they could finish the project at any
  195 +time. The deployment limitation was the substantial technical issue we could
  196 +tackle in the short term. As a result, we worked to deploy one version of the
  197 +project onto our infrastructure and showed it to the government evaluators. This
  198 +strategy proved them we could efficiently deliver new features, fulfill their
  199 +expectations regarding the delivery of the requirements, and incited them to
  200 +demand the last version working in the MP infrastructure. These results, in turn,
  201 +generated more pressure on the IT department responsible for the deployment
  202 +routines. With each CD cycle, we gradually built a new relationship among all
  203 +parties and, by the end of the project, we became active participants in the
  204 +deploy operations.
113 205  
114 206 ## Our Continuous Delivery Pipeline
115 207  
... ... @@ -169,7 +261,7 @@ five reasons:
169 261 * Packaging allows configurations and permissions control.
170 262  
171 263 After creating a new tag for one component, the developers informed the DevOps
172   -[3] team, and the packaging process began. A set of scripts fully automated the
  264 +[5] team, and the packaging process began. A set of scripts fully automated the
173 265 three packaging steps aforementioned. With all them running successfully, the
174 266 new packages would be ready to be used by our deployment scripts.
175 267  
... ... @@ -206,7 +298,7 @@ users.
206 298  
207 299 ## Benefits
208 300  
209   -Research points out many advantages of CD usage in the industry[2, 5], such as
  301 +Research points out many advantages of CD usage in the industry[2, 6], such as
210 302 accelerated time to market, building the right product, productivity and
211 303 efficiency improvements, stable releases, and better customer satisfaction.
212 304 Working with the government, we noticed the following additional benefits.
... ... @@ -355,10 +447,12 @@ We thank our colleagues, Nelson Lago, Lucas Kanashiro and Rafael Manzo, and this
355 447 3. ISO/IEC/IEEE International Standard - Systems and software engineering -- System life cycle processes," in ISO/IEC/IEEE 15288 First edition 2015-05-15, 2015, pp.1-118.
356 448 4. Meirelles, P., Wen, M., Terceiro, A., Siqueira, R., Kanashiro, L., and Neri, H. 2017. Brazilian Public Software Portal: an integrated platform for collaborative development. In Proceedings of the 13th International Symposium on Open Collaboration (OpenSym '17). ACM, Article 16, 10 pages.
357 449 5. Davis, Jennifer and Daniels, Katherine, Effective DevOps: building a culture of collaboration, affinity, and tooling at scale, 2016, " O'Reilly Media, Inc."
358   -6. Humble, J. and Farley, D., 2010. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Adobe Reader). Pearson Education.
359   -7. Savor, T., Douglas, M., Gentili, M., Williams, L., Beck, K. and Stumm, M., 2016, May. Continuous deployment at Facebook and OANDA. In Proceedings of the 38th International Conference on Software Engineering Companion (pp. 21-30). ACM.
  450 +6. Savor, T., Douglas, M., Gentili, M., Williams, L., Beck, K. and Stumm, M., 2016, May. Continuous deployment at Facebook and OANDA. In Proceedings of the 38th International Conference on Software Engineering Companion (pp. 21-30). ACM.
  451 +<!--- Referências soltas
  452 +7. Humble, J. and Farley, D., 2010. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Pearson Education.
360 453 8. Chen, L., 2015, May. Research opportunities in continuous delivery: reflections from two years' experiences in a large bookmaking company. In Proceedings of the Third International Workshop on Release Engineering (pp. 2-2). IEEE Press.
361 454 9. Chen, L., 2015, May. Towards architecting for continuous delivery. In Software Architecture (WICSA), 2015 12th Working IEEE/IFIP Conference on (pp. 131-134). IEEE.
  455 +--->
362 456  
363 457  
364 458  
... ...