Commit a2245409dc7f6e742cf967e0127fb81de2efdd56

Authored by Melissa Wen
2 parents d9e303aa 7c1ee6ef

Merge branch 'master' of http://softwarepublico.gov.br/gitlab/softwarepublico/articles

ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
... ... @@ -121,54 +121,48 @@ we moved to preparing the release.
121 121  
122 122 ### Preparing a new release
123 123  
124   -Our release process was divided in two perspectives in terms of git tags: the
  124 +Our release process was divided in two perspectives in terms of Git tags: the
125 125 application and the SPB Portal. The application tag refers to the specific
126 126 feature or bug fix and is a monotonically increasing. A new tag on any system
127 127 yielded a new SPB Portal tag.
128 128  
129 129 When all tests passed for a given component, we manually created a new
130 130 application tag for it. As a consequence, that automatically created a new tag
131   -for the SPB Portal. Notice that we have forks of the original softwares and, as
132   -consequence, we had different tag values.
  131 +for the SPB Portal. Notice that we forked of the original software projects
  132 +and, as consequence, we had different tag values.
133 133  
134 134 ### Packaging
135 135  
136   -The platform is running on the CentOS 7 GNU/Linux distribution.
137   -Basically, packaging a software for that distribution has three steps: write
138   -the script for the specific environment (RPM); build the package; and upload
139   -it to a package repository.
  136 +The platform is running on the CentOS 7 GNU/Linux distribution. Basically,
  137 +packaging a software for that distribution has three steps: write the script
  138 +for the specific environment (RPM), build the package, and upload it to a
  139 +package repository.
140 140  
141 141 We chose to create our own packages for each software component for several
142 142 reasons:
143   -* Not all software was packaged by the community;
144   -* And those that existed were outdated;
  143 +
  144 +* Not all software was packaged by the community and those that existed were
  145 +outdated;
145 146 * Packaging makes it easy to manage the software on a given distribution;
146 147 * It simplifies the deployment;
147 148 * Packaging follows the distribution’s best practices and,
148 149 * Allows configurations and permissions control.
149 150  
150 151 After creating a new tag for one component, the DevOps team was notified and
151   -packaging process began. In the normal case, the three packaging steps
152   -aforementioned were fully automated by a set of scripts.
153   -
154   -However, if the developers reported to DevOps any eventual dependency change,
155   -the first packaging step had to be manual. For instance, suppose one system
156   -starts requiring another system to be initialized first. That required the
157   -DevOps to manually update the packaging script respective to these systems.
158   -
159   -After all these scripts have run successfully, the new packages would be ready
160   -to use by our subsequent deployment scripts.
  152 +the packaging process began. In the normal case, the three packaging steps
  153 +aforementioned were fully automated by a set of scripts. With all these scripts
  154 +running successfully, the new packages would be ready to use by our subsequent
  155 +deployment scripts.
161 156  
162   -### Validation Environment
163   -
164   -[//]: # (TODO - Mencionar que a ferramenta era baseada em Chef - Dá um peso importante.)
  157 +### Validation Environment Deployment
165 158  
166 159 The Validation Environment (VE) is a replica of the Production Environment
167 160 (PE), with two exceptions: only the government officers and us had access to it
168   -and all the data is anonymised. To configure the environment, we use a
169   -configuration management tool. That maintained environment consistency
170   -simplifying the deployment process. Additionally, the packages we built on
171   -the last step were readily available to use by the management tool.
  161 +as well as all the data is anonymised. To configure the environment, we used
  162 +our configuration management tool: Chake (serverless configuration with Chef).
  163 +That maintained environment consistency simplifying the deployment process.
  164 +Additionally, the packages we built on the last step were readily available to
  165 +use by the management tool.
172 166  
173 167 The VE was used by the government agents to validate new features and required
174 168 changes. Also, the VE was useful to verify the integrity of the entire portal
... ... @@ -176,20 +170,21 @@ as part of the next step in the pipeline.
176 170  
177 171 ### Acceptance Tests
178 172  
179   -After we completely deploy a new SPB Portal version in the VE, the government agents
180   -are responsible for checking features and/or bug fixes required by them. If the
181   -technicians identify a problem, they notify the developers. These problems are
182   -fixed and the pipeline restarts from scratch. If everything is validated, we
183   -move forward.
  173 +After we completely deploy, a new SPB Portal version in the VE, the government
  174 +agents are responsible for checking features and bug fixes required by them. If
  175 +the technicians identify a problem, they notified the developers via comments
  176 +on an git issue related to the user story (features) already registered in our
  177 +Gitlab at the SPB Portal. These problems were fixed and the pipeline restarted
  178 +from scratch. If everything is validated, we moved forward.
184 179  
185   -### Production Deployment
  180 +### Production Environment Deployment
186 181  
187   -After the government finish the VE check, it is cleared for deployment and we
188   -can finally begin the deployment to Production Environment (PE). For this we
189   -use the same configuration management tool as in the VE as well with same
190   -scripts and package versions. After the deploy is completed, both VE and PE
191   -are running identical software. This is the point where new features and bug
192   -fixes are finally available to end users.
  182 +After the government finished the VE check, it was cleared for deployment and
  183 +we could finally began the deployment to Production Environment (PE). For this
  184 +we also used our configuration management tool as in the VE as well with same
  185 +scripts and package versions. After the deploy was completed, both VE and PE
  186 +were running identical software. This was the point where new features and bug
  187 +fixes were finally available to the end users.
193 188  
194 189 ## Benefits
195 190  
... ... @@ -270,25 +265,24 @@ project management.
270 265  
271 266 ## Challenges
272 267  
273   -We successfully built a functional CD pipeline. In the end, we took over the
274   -deployment process from the government. That allowed us to survive into an
275   -unstable political scenario. However, we recognize that many challenges still
276   -need to be addressed by the industry and academia together.
  268 +We successfully built a CD pipeline. In the end, we took over the deployment
  269 +process from the government. That allowed us to survive into an unstable
  270 +political scenario. However, we recognize that many challenges still need to be
  271 +addressed by the industry and academia together.
277 272  
278 273 ### Build CD from scratch
279   -
  274 +
280 275 Taking on CD responsibilities had a significant impact on the team. We did not
281 276 have the know-how and had little time to come up with a working pipeline. The
282   -seniors were crucial at this point. They came up with an initial solution to
283   -get us started. That already enabled us to automatize the deploy, even though
284   -the process was still rudimentary. We had to evolve our solution on-the-fly. We
285   -dedicated a few developers to this task.
  277 +senior developers were crucial at this point. They came up with an initial
  278 +solution to get us started. That already enabled us to automatize the deploy,
  279 +even though the process was still rudimentary. We had to evolve our solution
  280 +on-the-fly. We dedicated a few developers to this task.
286 281  
287   -Building a CD pipeline was hard in the beginning, and we succeed thanks to the
288   -seniors experience. We believe that more tools that provide out-of-the-box
289   -standardized CD pipelines would be of great help for inexperienced teams. Tools
290   -that track each step of the pipeline and organize logs in a human-manageable
291   -way are necessary too.
  282 +Building a CD pipeline was hard in the beginning. We believe that more tools
  283 +that provide out-of-the-box standardized CD pipelines would be of great help
  284 +for inexperienced teams. Tools that track each step of the pipeline and
  285 +organize logs in a human-manageable way are necessary too.
292 286  
293 287 ### Handling inexperienced teams
294 288  
... ... @@ -303,14 +297,12 @@ are needed.
303 297 In the project’s first half we struggled with deploy related problems in the
304 298 government structure. We were in a paradoxical situation. The government
305 299 demanded speedy deliveries but would not give access to their production
306   -infrastructure. As an example, only in specific situations the government
307   -allowed us to access the PE. It took some interactions with government agents
308   -before we convinced them to create the VE as an isolated replica of the PE in
309   -their own infrastructure. The government agents then realized that it could be
310   -beneficial for the project if they granted us access to part of the structure.
311   -We believe it is required more research on development protocols and policies
312   -to improve the relation between industry and government, specially regarding
313   -CD.
  300 +infrastructure. Afterwards some interactions with government agents, they
  301 +created the VE as an isolated replica of the PE in their own infrastructure.
  302 +The government agents then realised that it could be beneficial for the project
  303 +if they granted us access to part of the structure. We believe it is required
  304 +more research on development protocols and policies to improve the relation
  305 +between industry and government, specially regarding CD.
314 306  
315 307 ## References
316 308  
... ...