Commit c1aeba3c32717ca75a3d8656877cc4c268d81285
1 parent
f537aec0
Exists in
master
and in
3 other branches
[ieeeSW] Improving SPB technical presentation and minor fixes
Showing
1 changed file
with
109 additions
and
116 deletions
Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
... | ... | @@ -72,21 +72,21 @@ benefits [2]. |
72 | 72 | |
73 | 73 | SPB is a governmental program created to foster sharing and collaboration on |
74 | 74 | Open Source Software (OSS) development for the Brazilian public administration. |
75 | -In their own projects, the Ministry managed both software requirements and server | |
76 | -infrastructure. However, its hierarchical and traditional processes made them | |
77 | -unfamiliar with new software development techniques, such as CD. Any of our | |
78 | -requests had to pass through layers of bureaucracy before being answered; | |
75 | +In their own projects, the Ministry managed both software requirements and | |
76 | +server infrastructure. However, its hierarchical and traditional processes made | |
77 | +them unfamiliar with new software development techniques, such as CD. Any of | |
78 | +our requests had to pass through layers of bureaucracy before being answered; | |
79 | 79 | accessing their infrastructure to deploy updated software was not different. |
80 | 80 | The difficulties were aggravated because the new SPB portal is an unprecedented |
81 | 81 | platform in the Brazilian government, with a complicated deployment process. |
82 | 82 | |
83 | -The project suffered significant interference from the | |
84 | -board of directors throughout time because the portal represents an interface between | |
85 | -government and society. In light of political interests, directors continually | |
86 | -imposed changes to the platform while ignoring our technical advice. In 2015, | |
87 | -the board of directors was changed and, with it, the vision of the project. New | |
88 | -directors had different political agendas which affected the project's | |
89 | -requirements previously approved. | |
83 | +The project suffered significant interference from the board of directors | |
84 | +throughout time because the portal represents an interface between government | |
85 | +and society. In light of political interests, directors continually imposed | |
86 | +changes to the platform while ignoring our technical advice. In 2015, the board | |
87 | +of directors was changed and, with it, the vision of the project. New directors | |
88 | +had different political agendas which affected the project's requirements | |
89 | +previously approved. | |
90 | 90 | |
91 | 91 | <!--- |
92 | 92 | mudar em todas as ocorrências de agents por staff, mas checar com o Fabio. |
... | ... | @@ -96,22 +96,24 @@ In this context, we overcame two distinct challenges: (1) deconstructing |
96 | 96 | the widespread belief among government staff that any project in partnership |
97 | 97 | with a University is doomed to fail, and (2) dealing with bureaucracies |
98 | 98 | involved in the deployment process. |
99 | + | |
99 | 100 | <!-- Melissa: Acho que no segundo, seria bom falar: lidar com a aritmia que as burocracias do governo causavam ao nosso processo de deploy? --> |
100 | 101 | |
101 | 102 | Firstly, our team was not from a typical company, consisting mainly of |
102 | -undergraduate students supported senior developers and designers, all coordinated by two professors. Accordingly, time and | |
103 | -resources allocation, accountability, and team continuity might be construed | |
104 | -as "unprofessional." On the government side, the SPB portal evolution was the | |
105 | -first software development collaboration between universities and the Ministry | |
106 | -staff involved, raising disbelief. | |
107 | - | |
108 | -Secondly, our team approached software deployment differently from the Ministry. | |
109 | -We believed frequent delivery is better for the project’s success. In contrast, | |
110 | -the Ministry is used to the idea of a single deployment at the end of the | |
111 | -project, and neither their bureaucratic structure nor their technical expertise | |
112 | -were conducive with this style of work. That ended up hampering the benefits of | |
113 | -the tool and preventing us from showing off the fruits of the project to those | |
114 | -responsible for evaluating it. | |
103 | +undergraduate students supported senior developers and designers, all | |
104 | +coordinated by two professors. Accordingly, time and resources allocation, | |
105 | +accountability, and team continuity might be construed as "unprofessional". On | |
106 | +the government side, the SPB portal evolution was the first software | |
107 | +development collaboration between universities and the Ministry staff involved, | |
108 | +raising disbelief. | |
109 | + | |
110 | +Secondly, our team approached software deployment differently from the | |
111 | +Ministry. We believed frequent delivery is better for the project’s success. | |
112 | +In contrast, the Ministry is used to the idea of a single deployment at the end | |
113 | +of the project, and neither their bureaucratic structure nor their technical | |
114 | +expertise were conducive with this style of work. That ended up hampering the | |
115 | +benefits of the tool and preventing us from showing off the fruits of the | |
116 | +project to those responsible for evaluating it. | |
115 | 117 | |
116 | 118 | <!-- Melissa: tentar dividir esse último parágrafo em 2? Tá cansativo ler ele todo - não consigo respirar :P --> |
117 | 119 | |
... | ... | @@ -123,10 +125,10 @@ deploy one version of the project onto our infrastructure and showed it to the |
123 | 125 | government evaluators. This strategy proved them we could efficiently provide |
124 | 126 | new features, fulfill their expectations regarding the delivery of the |
125 | 127 | requirements, and incited them to demand the latest version to be deployed in |
126 | -the Ministry infrastructure. This efficiency generated more pressure on the IT department | |
127 | -responsible for the deployment routines. With each CD cycle, we gradually built | |
128 | -a new relationship among all parties and, by the end of the project, we became | |
129 | -active participants in the deploy operations. | |
128 | +the Ministry infrastructure. This efficiency generated more pressure on the IT | |
129 | +department responsible for the deployment routines. With each CD cycle, we | |
130 | +gradually built a new relationship among all parties and, by the end of the | |
131 | +project, we became active participants in the deploy operations. | |
130 | 132 | |
131 | 133 | <!-- |
132 | 134 | In |
... | ... | @@ -137,14 +139,13 @@ team shrinked to 20 students, one designer, and two developers. |
137 | 139 | --> |
138 | 140 | ## Our Continuous Delivery Pipeline |
139 | 141 | |
140 | -SPB portal is a CDE with additional social features. We built | |
141 | -system-of-systems, adapting and integrating five existing OSS projects: Colab | |
142 | +SPB portal is a CDE with additional social features. We built system-of-systems | |
143 | +adapting and integrating five existing OSS projects: Colab | |
142 | 144 | (www.github.com/colab), Noosfero (www.noosfero.org), Gitlab (www.gitlab.com), |
143 | -Mezuro (www.mezuro.org), and Mailman (www.list.org). All integrated system | |
144 | -represents a total of more than 106.253 commits and 1.347.421 lines of code. | |
145 | -That solution orchestrated these multiple components and allowed us to smoothly | |
146 | -provide a unified interface for final users, including single sign-on and | |
147 | -global searches [1]. | |
145 | +Mezuro (www.mezuro.org), and Mailman (www.list.org). Colab orchestrates these | |
146 | +multiple components and allowed us to smoothly provide a unified interface for | |
147 | +final users, including single sign-on and global searches [1]. All integrated | |
148 | +system represents a total of 106.253 commits and 1.347.421 lines of code. | |
148 | 149 | |
149 | 150 |  |
150 | 151 | |
... | ... | @@ -154,25 +155,24 @@ practices, as presented in Figure 1. |
154 | 155 | |
155 | 156 | ### Automated Tests |
156 | 157 | |
157 | -Each integrated system had to be tested with its own test suite. This checking was not | |
158 | -enough, however: we even had to test the platform as a whole. Given that the | |
159 | -plugins also have their own test suites and these suites assume a double role | |
160 | -as both plugin tests and as integration tests. | |
161 | -If any test suite failed, by either a test error or coverage | |
162 | -reduction below a certain threshold, the process stopped. Only when all tests | |
163 | -passed, the pipeline proceeded to the step of release preparation. | |
158 | +Each integrated system had to be tested with its own test suite. This checking | |
159 | +was not enough, however: we even had to test the platform as a whole. Given | |
160 | +that the plugins also have their own test suites and these suites assume a | |
161 | +double role as both plugin tests and as integration tests. If any test suite | |
162 | +failed, by either a test error or coverage reduction below a certain threshold, | |
163 | +the process stopped. Only when all tests passed, the pipeline proceeded to the | |
164 | +step of release preparation. | |
164 | 165 | |
165 | 166 | ### Preparing a New Release |
166 | 167 | |
167 | 168 | Each software component was hosted in a separate Git repository. A new release |
168 | -of a component was tagged referencing a specific new feature or bug | |
169 | -fix. SPB had its own Git repository | |
170 | -(www.softwarepublico.gov.br/gitlab/softwarepublico). An SPB portal release was | |
171 | -an aggregate of versions of all of its components. When a new component release | |
172 | -passed all of the SPB integration tests, we manually created a corresponding | |
173 | -new tag in its repository. Therefore, a new tag on any software component | |
174 | -yielded a new SPB portal release. At the end of this process, we started | |
175 | -packaging. | |
169 | +of a component was tagged referencing a specific new feature or bug fix. SPB | |
170 | +had its own Git repository (www.softwarepublico.gov.br/gitlab/softwarepublico). | |
171 | +An SPB portal release was an aggregate of versions of all of its components. | |
172 | +When a new component release passed all of the SPB integration tests, we | |
173 | +manually created a corresponding new tag in its repository. Therefore, a new | |
174 | +tag on any software component yielded a new SPB portal release. At the end of | |
175 | +this process, we started packaging. | |
176 | 176 | |
177 | 177 | ### Packaging |
178 | 178 | |
... | ... | @@ -202,19 +202,18 @@ the next step in the pipeline. |
202 | 202 | |
203 | 203 | ### Acceptance Tests |
204 | 204 | |
205 | -After a new SPB portal deployment in the VE, the Ministry was | |
206 | -responsible for checking the required features and bug fixes. If they | |
207 | -identified a problem, they would notify the developers via | |
208 | -comments on the SPB portal's issue tracker, prompting the team to fix | |
209 | -it and restart the pipeline. Otherwise, we could move forward. | |
205 | +After a new SPB portal deployment in the VE, the Ministry was responsible for | |
206 | +checking the required features and bug fixes. If they identified a problem, | |
207 | +they would notify the developers via comments on the SPB portal's issue | |
208 | +tracker, prompting the team to fix it and restart the pipeline. Otherwise, we | |
209 | +could move forward. | |
210 | 210 | |
211 | 211 | ### Production Environment Deployment |
212 | 212 | |
213 | -After the VE check, we could finally begin the | |
214 | -deployment in the PE, with the same configuration management tool, | |
215 | -scripts, and package versions as in the VE. After the deploy was completed, | |
216 | -both VE and PE were identical. At that point, new features and bug | |
217 | -fixes were finally available to end users. | |
213 | +After the VE check, we could finally begin the deployment in the PE, with the | |
214 | +same configuration management tool, scripts, and package versions as in the VE. | |
215 | +After the deploy was completed, both VE and PE were identical. At that point, | |
216 | +new features and bug fixes were finally available to end users. | |
218 | 217 | |
219 | 218 | ## Benefits |
220 | 219 | |
... | ... | @@ -227,8 +226,9 @@ project and associates them with the creation of a DevOps team. |
227 | 226 |  |
228 | 227 | |
229 | 228 | In the time of 30 months, we deployed a total of 84 versions. Over 64% of these |
230 | -activities happened in the second half of 2015, with the DevOps team creation. Even with the reduction of the team as a whole, in the last months of | |
231 | -the project, this pace of deployment was maintained until we finished all our | |
229 | +activities happened in the second half of 2015, with the DevOps team creation. | |
230 | +Even with the reduction of the team as a whole, in the last months of the | |
231 | +project, this pace of deployment was maintained until we finished all our | |
232 | 232 | activities. |
233 | 233 | |
234 | 234 | Besides these results, working with the government, we noticed the following |
... | ... | @@ -236,33 +236,33 @@ additional benefits. |
236 | 236 | |
237 | 237 | ### Strengthening Trust in the Relationship with the Government |
238 | 238 | |
239 | -CD helped strengthen trust in the relationship between developers and | |
240 | -the Ministry staff. Before using CD, they could validate features | |
241 | -developed only at the end of the release cycle, usually every four months. | |
242 | -With the implementation of CD, intermediate and candidate versions became | |
243 | -available, allowing them to perform small validations over time. | |
244 | -Constant monitoring of the development work brought greater security to the | |
245 | -Ministry leaders and improved the interactions with our team. | |
239 | +CD helped strengthen trust in the relationship between developers and the | |
240 | +Ministry staff. Before using CD, they could validate features developed only at | |
241 | +the end of the release cycle, usually every four months. With the | |
242 | +implementation of CD, intermediate and candidate versions became available, | |
243 | +allowing them to perform small validations over time. Constant monitoring of | |
244 | +the development work brought greater security to the Ministry leaders and | |
245 | +improved the interactions with our team. | |
246 | 246 | |
247 | 247 | ### Responsiveness to Change |
248 | 248 | |
249 | 249 | Responsiveness was one of the direct benefits of adopting the CD pipeline. |
250 | 250 | Political interests may change requirements and priorities at any time. The |
251 | 251 | ability to react quickly to changes requested by the Ministry was vital to the |
252 | -project’s survival for 30 months. We noticed that if we took too | |
253 | -long to meet their demands, they would threaten to reduce financial support and | |
254 | -even cancel the project. | |
252 | +project’s survival for 30 months. We noticed that if we took too long to meet | |
253 | +their demands, they would threaten to reduce financial support and even cancel | |
254 | +the project. | |
255 | 255 | |
256 | 256 | CD helped us keep the PE up-to-date, even with partial versions of a feature. |
257 | -Therefore, we always had something to show on meetings, easing their | |
258 | -concerns about the final delivery of the platform. For our team, CD made | |
259 | -developers more confident that the project would last a little longer. | |
257 | +Therefore, we always had something to show on meetings, easing their concerns | |
258 | +about the final delivery of the platform. For our team, CD made developers more | |
259 | +confident that the project would last a little longer. | |
260 | 260 | |
261 | 261 | ### Shared Responsibility |
262 | 262 | |
263 | 263 | According to the conventional Ministry process, the development team could not |
264 | -track what happened to the code after its delivery, since their staff was | |
265 | -the only ones responsible for deployment. The implementation of CD made our | |
264 | +track what happened to the code after its delivery, since their staff was the | |
265 | +only ones responsible for deployment. The implementation of CD made our | |
266 | 266 | development team feel equally responsible for what was getting into production |
267 | 267 | and take ownership of the project. |
268 | 268 | |
... | ... | @@ -290,18 +290,11 @@ Fabio sugeriu Lessons Learned, mas vamos mostrar exemplos da revista para ele ol |
290 | 290 | ## Lessons Learned |
291 | 291 | |
292 | 292 | Due to the successful building of the CD pipeline, we improved the Ministry |
293 | -deployment process and kept the project alive. We now look at the lessons learned. | |
293 | +deployment process and kept the project alive. We now look at the lessons | |
294 | +learned. | |
294 | 295 | |
295 | 296 | ### Build CD From Scratch |
296 | 297 | |
297 | -<!--- | |
298 | -We started with a team composed of 24 undergraduate students, one designer, and | |
299 | -two senior developers. In 2015, our team grew to 36 students, two designers, | |
300 | -eight senior developers. For 2016, due to budget constraints, our team shrinked | |
301 | -to 20 students, one designer, and two developers; however, in the last 3 | |
302 | -months, only 6 students closed the project. | |
303 | ---> | |
304 | - | |
305 | 298 | Taking on the responsibility for implementing CD impacted the whole team. Most |
306 | 299 | of our team members did not have CD know-how, and we had few working hours |
307 | 300 | available to build the pipeline. The construction and maintenance of the CD |
... | ... | @@ -318,45 +311,45 @@ deployment process. The solution enabled us to automate the deployment, even |
318 | 311 | though the process was, initially, still rudimentary. |
319 | 312 | |
320 | 313 | 2. _Interchange team members and encourage teammates to migrate to the DevOps |
321 | -team._ The benefits were twofold: mitigating the difficulty | |
322 | -in sharing knowledge between DevOps developers and feature developers, and | |
323 | -evolving the process on-the-fly. | |
314 | +team._ The benefits were twofold: mitigating the difficulty in sharing | |
315 | +knowledge between DevOps developers and feature developers, and evolving the | |
316 | +process on-the-fly. | |
324 | 317 | |
325 | 318 | ### Overcoming Mistrust |
326 | 319 | |
327 | 320 | <!-- Precisa 'dessuavizar' o porque das mudanças de requisitos - a motivação para ela não era tão natural como as que ocorrem em métodos ágeis --> |
328 | 321 | Adopting an unfamiliar approach requires trust. In the Ministry, traditionally, |
329 | 322 | a software was the product delivered at the end of a development contract; they |
330 | -expected and were prepared to validate and deploy a single deliverable. This process | |
331 | -was not adequate for the SPB: because the SPB portal is a system-of-systems, | |
332 | -the steady growth of its complexity made large deliveries unsustainable. | |
333 | -Therefore, the CD approach was necessary, but how | |
334 | -to build trust and gain autonomy to implement a process that was not yet part | |
335 | -of the dynamics of the Ministry? | |
323 | +expected and were prepared to validate and deploy a single deliverable. This | |
324 | +process was not adequate for the SPB: because the SPB portal is a | |
325 | +system-of-systems, the steady growth of its complexity made large deliveries | |
326 | +unsustainable. Therefore, the CD approach was necessary, but how to build | |
327 | +trust and gain autonomy to implement a process that was not yet part of the | |
328 | +dynamics of the Ministry? | |
336 | 329 | |
337 | 330 | 1. _Demonstrate actual results, do not simply tell._ Initially, we did not have |
338 | 331 | access to the Ministry infrastructure, so we created our own validation |
339 | -environment. Thus, we were able to follow the CD pipeline until | |
340 | -production deployment, when we faced two problems. First, our pace of | |
341 | -intermediate deliveries to the government was faster than the deployment to | |
342 | -production by the Ministry staff. Second, specific issues of the Ministry | |
343 | -infrastructure made some validated features not work as expected in the PE. | |
344 | -That situation gave us arguments to negotiate access to the PE. | |
345 | - | |
346 | -2. _Make project management transparent and collaborative for government staff._ | |
347 | -Allowing the Ministry staff to track our development process showed them | |
348 | -we were fulfilling our commitments. They started to interact more actively in | |
349 | -the generation of versions and became involved in the CD. After understanding it, | |
350 | -the Ministry staff helped us negotiate access to a VE with the Ministry leaders, | |
351 | -creating as an isolated replica of the PE. | |
352 | - | |
353 | -3. _Gain the confidence of government staff._ With the PE replica, we | |
354 | -were able to run the entire pipeline and win the trust of the Ministry staff | |
355 | -involved in the process. They saw the mobilization and responsiveness of our | |
356 | -team to generate each new version package. They also recognized the quality of our | |
357 | -work and deployment process. In the end, the Ministry staff realized | |
358 | -that it would be beneficial for the project if they granted us access to the | |
359 | -infrastructure, both VE, and PE. | |
332 | +environment. Thus, we were able to follow the CD pipeline until production | |
333 | +deployment, when we faced two problems. First, our pace of intermediate | |
334 | +deliveries to the government was faster than the deployment to production by | |
335 | +the Ministry staff. Second, specific issues of the Ministry infrastructure made | |
336 | +some validated features not work as expected in the PE. That situation gave us | |
337 | +arguments to negotiate access to the PE. | |
338 | + | |
339 | +2. _Make project management transparent and collaborative for government | |
340 | +staff._ Allowing the Ministry staff to track our development process showed | |
341 | +them we were fulfilling our commitments. They started to interact more actively | |
342 | +in the generation of versions and became involved in the CD. After | |
343 | +understanding it, the Ministry staff helped us negotiate access to a VE with | |
344 | +the Ministry leaders, creating as an isolated replica of the PE. | |
345 | + | |
346 | +3. _Gain the confidence of government staff._ With the PE replica, we were able | |
347 | +to run the entire pipeline and win the trust of the Ministry staff involved in | |
348 | +the process. They saw the mobilization and responsiveness of our team to | |
349 | +generate each new version package. They also recognized the quality of our work | |
350 | +and deployment process. In the end, the Ministry staff realized that it would | |
351 | +be beneficial for the project if they granted us access to the infrastructure, | |
352 | +both VE, and PE. | |
360 | 353 | |
361 | 354 | ## References |
362 | 355 | ... | ... |