Merge branch 'master' of http://softwarepublico.gov.br/gitlab/softwarepublico/articles

Melissa Wen
2 parents d9e303aa 7c1ee6ef
Showing 1 changed file with 52 additions and 60 deletions Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
@@ -121,54 +121,48 @@ we moved to preparing the release.
  
 ### Preparing a new release
  
-Our release process was divided in two perspectives in terms of git tags: the
+Our release process was divided in two perspectives in terms of Git tags: the
 application and the SPB Portal. The application tag refers to the specific
 feature or bug fix and is a monotonically increasing. A new tag on any system
 yielded a new SPB Portal tag.
  
 When all tests passed for a given component, we manually created a new
 application tag for it. As a consequence, that automatically created a new tag
-for the SPB Portal. Notice that we have forks of the original softwares and, as
-consequence, we had different tag values.
+for the SPB Portal. Notice that we forked of the original software projects
+and, as consequence, we had different tag values.
  
 ### Packaging
  
-The platform is running on the CentOS 7 GNU/Linux distribution.
-Basically, packaging a software for that distribution has three steps: write
-the script for the specific environment (RPM); build the package; and upload
-it to a package repository.
+The platform is running on the CentOS 7 GNU/Linux distribution.  Basically,
+packaging a software for that distribution has three steps: write the script
+for the specific environment (RPM), build the package, and upload it to a
+package repository.
  
 We chose to create our own packages for each software component for several
 reasons:
-* Not all software was packaged by the community;
-* And those that existed were outdated;
+
+* Not all software was packaged by the community and those that existed were
+outdated;
 * Packaging makes it easy to manage the software on a given distribution;
 * It simplifies the deployment;
 * Packaging follows the distribution’s best practices and,
 * Allows configurations and permissions control.
  
 After creating a new tag for one component, the DevOps team was notified and
-packaging process began. In the normal case, the three packaging steps
-aforementioned were fully automated by a set of scripts.
-
-However, if the developers reported to DevOps any eventual dependency change,
-the first packaging step had to be manual. For instance, suppose one system
-starts requiring another system to be initialized first. That required the
-DevOps to manually update the packaging script respective to these systems.
-
-After all these scripts have run successfully, the new packages would be ready
-to use by our subsequent deployment scripts.
+the packaging process began. In the normal case, the three packaging steps
+aforementioned were fully automated by a set of scripts. With all these scripts
+running successfully, the new packages would be ready to use by our subsequent
+deployment scripts.
  
-### Validation Environment
-
-[//]: # (TODO - Mencionar que a ferramenta era baseada em Chef - Dá um peso importante.)
+### Validation Environment Deployment
  
 The Validation Environment (VE) is a replica of the Production Environment
 (PE), with two exceptions: only the government officers and us had access to it
-and all the data is anonymised. To configure the environment, we use a
-configuration management tool. That maintained environment consistency
-simplifying the deployment process. Additionally, the packages we built on
-the last step were readily available to use by the management tool.
+as well as all the data is anonymised. To configure the environment, we used
+our configuration management tool: Chake (serverless configuration with Chef).
+That maintained environment consistency simplifying the deployment process.
+Additionally, the packages we built on the last step were readily available to
+use by the management tool.
  
 The VE was used by the government agents to validate new features and required
 changes. Also, the VE was useful to verify the integrity of the entire portal
@@ -176,20 +170,21 @@ as part of the next step in the pipeline.
  
 ### Acceptance Tests
  
-After we completely deploy a new SPB Portal version in the VE, the government agents
-are responsible for checking features and/or bug fixes required by them. If the
-technicians identify a problem, they notify the developers. These problems are
-fixed and the pipeline restarts from scratch. If everything is validated, we
-move forward.
+After we completely deploy, a new SPB Portal version in the VE, the government
+agents are responsible for checking features and bug fixes required by them. If
+the technicians identify a problem, they notified the developers via comments
+on an git issue related to the user story (features) already registered in our
+Gitlab at the SPB Portal. These problems were fixed and the pipeline restarted
+from scratch. If everything is validated, we moved forward.
  
-### Production Deployment
+### Production Environment Deployment
  
-After the government finish the VE check, it is cleared for deployment and we
-can finally begin the deployment to Production Environment (PE). For this we
-use the same configuration management tool as in the VE as well with same
-scripts and package versions. After the deploy is completed, both VE and PE
-are running identical software. This is the point where new features and bug
-fixes are finally available to end users.
+After the government finished the VE check, it was cleared for deployment and
+we could finally began the deployment to Production Environment (PE). For this
+we also used our configuration management tool as in the VE as well with same
+scripts and package versions. After the deploy was completed, both VE and PE
+were running identical software. This was the point where new features and bug
+fixes were finally available to the end users.
  
 ## Benefits
  
@@ -270,25 +265,24 @@ project management.
  
 ## Challenges
  
-We successfully built a functional CD pipeline. In the end, we took over the
-deployment process from the government. That allowed us to survive into an
-unstable political scenario. However, we recognize that many challenges still
-need to be addressed by the industry and academia together.
+We successfully built a CD pipeline. In the end, we took over the deployment
+process from the government. That allowed us to survive into an unstable
+political scenario. However, we recognize that many challenges still need to be
+addressed by the industry and academia together.
  
 ### Build CD from scratch
-
+	
 Taking on CD responsibilities had a significant impact on the team. We did not
 have the know-how and had little time to come up with a working pipeline. The
-seniors were crucial at this point. They came up with an initial solution to
-get us started. That already enabled us to automatize the deploy, even though
-the process was still rudimentary. We had to evolve our solution on-the-fly. We
-dedicated a few developers to this task.
+senior developers were crucial at this point. They came up with an initial
+solution to get us started. That already enabled us to automatize the deploy,
+even though the process was still rudimentary. We had to evolve our solution
+on-the-fly. We dedicated a few developers to this task.
  
-Building a CD pipeline was hard in the beginning, and we succeed thanks to the
-seniors experience. We believe that more tools that provide out-of-the-box
-standardized CD pipelines would be of great help for inexperienced teams. Tools
-that track each step of the pipeline and organize logs in a human-manageable
-way are necessary too.
+Building a CD pipeline was hard in the beginning. We believe that more tools
+that provide out-of-the-box standardized CD pipelines would be of great help
+for inexperienced teams. Tools that track each step of the pipeline and
+organize logs in a human-manageable way are necessary too.
  
 ### Handling inexperienced teams
  
@@ -303,14 +297,12 @@ are needed.
 In the project’s first half we struggled with deploy related problems in the
 government structure. We were in a paradoxical situation. The government
 demanded speedy deliveries but would not give access to their production
-infrastructure. As an example, only in specific situations the government
-allowed us to access the PE. It took some interactions with government agents
-before we convinced them to create the VE as an isolated replica of the PE in
-their own infrastructure. The government agents then realized that it could be
-beneficial for the project if they granted us access to part of the structure.
-We believe it is required more research on development protocols and policies
-to improve the relation between industry and government, specially regarding
-CD.
+infrastructure. Afterwards some interactions with government agents, they
+created the VE as an isolated replica of the PE in their own infrastructure.
+The government agents then realised that it could be beneficial for the project
+if they granted us access to part of the structure.  We believe it is required
+more research on development protocols and policies to improve the relation
+between industry and government, specially regarding CD.
  
 ## References
...	...	@@ -121,54 +121,48 @@ we moved to preparing the release.
121	121
122	122	### Preparing a new release
123	123
124		-Our release process was divided in two perspectives in terms of git tags: the
	124	+Our release process was divided in two perspectives in terms of Git tags: the
125	125	application and the SPB Portal. The application tag refers to the specific
126	126	feature or bug fix and is a monotonically increasing. A new tag on any system
127	127	yielded a new SPB Portal tag.
128	128
129	129	When all tests passed for a given component, we manually created a new
130	130	application tag for it. As a consequence, that automatically created a new tag
131		-for the SPB Portal. Notice that we have forks of the original softwares and, as
132		-consequence, we had different tag values.
	131	+for the SPB Portal. Notice that we forked of the original software projects
	132	+and, as consequence, we had different tag values.
133	133
134	134	### Packaging
135	135
136		-The platform is running on the CentOS 7 GNU/Linux distribution.
137		-Basically, packaging a software for that distribution has three steps: write
138		-the script for the specific environment (RPM); build the package; and upload
139		-it to a package repository.
	136	+The platform is running on the CentOS 7 GNU/Linux distribution. Basically,
	137	+packaging a software for that distribution has three steps: write the script
	138	+for the specific environment (RPM), build the package, and upload it to a
	139	+package repository.
140	140
141	141	We chose to create our own packages for each software component for several
142	142	reasons:
143		-* Not all software was packaged by the community;
144		-* And those that existed were outdated;
	143	+
	144	+* Not all software was packaged by the community and those that existed were
	145	+outdated;
145	146	* Packaging makes it easy to manage the software on a given distribution;
146	147	* It simplifies the deployment;
147	148	* Packaging follows the distribution’s best practices and,
148	149	* Allows configurations and permissions control.
149	150
150	151	After creating a new tag for one component, the DevOps team was notified and
151		-packaging process began. In the normal case, the three packaging steps
152		-aforementioned were fully automated by a set of scripts.
153		-
154		-However, if the developers reported to DevOps any eventual dependency change,
155		-the first packaging step had to be manual. For instance, suppose one system
156		-starts requiring another system to be initialized first. That required the
157		-DevOps to manually update the packaging script respective to these systems.
158		-
159		-After all these scripts have run successfully, the new packages would be ready
160		-to use by our subsequent deployment scripts.
	152	+the packaging process began. In the normal case, the three packaging steps
	153	+aforementioned were fully automated by a set of scripts. With all these scripts
	154	+running successfully, the new packages would be ready to use by our subsequent
	155	+deployment scripts.
161	156
162		-### Validation Environment
163		-
164		-[//]: # (TODO - Mencionar que a ferramenta era baseada em Chef - Dá um peso importante.)
	157	+### Validation Environment Deployment
165	158
166	159	The Validation Environment (VE) is a replica of the Production Environment
167	160	(PE), with two exceptions: only the government officers and us had access to it
168		-and all the data is anonymised. To configure the environment, we use a
169		-configuration management tool. That maintained environment consistency
170		-simplifying the deployment process. Additionally, the packages we built on
171		-the last step were readily available to use by the management tool.
	161	+as well as all the data is anonymised. To configure the environment, we used
	162	+our configuration management tool: Chake (serverless configuration with Chef).
	163	+That maintained environment consistency simplifying the deployment process.
	164	+Additionally, the packages we built on the last step were readily available to
	165	+use by the management tool.
172	166
173	167	The VE was used by the government agents to validate new features and required
174	168	changes. Also, the VE was useful to verify the integrity of the entire portal
...	...	@@ -176,20 +170,21 @@ as part of the next step in the pipeline.
176	170
177	171	### Acceptance Tests
178	172
179		-After we completely deploy a new SPB Portal version in the VE, the government agents
180		-are responsible for checking features and/or bug fixes required by them. If the
181		-technicians identify a problem, they notify the developers. These problems are
182		-fixed and the pipeline restarts from scratch. If everything is validated, we
183		-move forward.
	173	+After we completely deploy, a new SPB Portal version in the VE, the government
	174	+agents are responsible for checking features and bug fixes required by them. If
	175	+the technicians identify a problem, they notified the developers via comments
	176	+on an git issue related to the user story (features) already registered in our
	177	+Gitlab at the SPB Portal. These problems were fixed and the pipeline restarted
	178	+from scratch. If everything is validated, we moved forward.
184	179
185		-### Production Deployment
	180	+### Production Environment Deployment
186	181
187		-After the government finish the VE check, it is cleared for deployment and we
188		-can finally begin the deployment to Production Environment (PE). For this we
189		-use the same configuration management tool as in the VE as well with same
190		-scripts and package versions. After the deploy is completed, both VE and PE
191		-are running identical software. This is the point where new features and bug
192		-fixes are finally available to end users.
	182	+After the government finished the VE check, it was cleared for deployment and
	183	+we could finally began the deployment to Production Environment (PE). For this
	184	+we also used our configuration management tool as in the VE as well with same
	185	+scripts and package versions. After the deploy was completed, both VE and PE
	186	+were running identical software. This was the point where new features and bug
	187	+fixes were finally available to the end users.
193	188
194	189	## Benefits
195	190
...	...	@@ -270,25 +265,24 @@ project management.
270	265
271	266	## Challenges
272	267
273		-We successfully built a functional CD pipeline. In the end, we took over the
274		-deployment process from the government. That allowed us to survive into an
275		-unstable political scenario. However, we recognize that many challenges still
276		-need to be addressed by the industry and academia together.
	268	+We successfully built a CD pipeline. In the end, we took over the deployment
	269	+process from the government. That allowed us to survive into an unstable
	270	+political scenario. However, we recognize that many challenges still need to be
	271	+addressed by the industry and academia together.
277	272
278	273	### Build CD from scratch
279		-
	274	+
280	275	Taking on CD responsibilities had a significant impact on the team. We did not
281	276	have the know-how and had little time to come up with a working pipeline. The
282		-seniors were crucial at this point. They came up with an initial solution to
283		-get us started. That already enabled us to automatize the deploy, even though
284		-the process was still rudimentary. We had to evolve our solution on-the-fly. We
285		-dedicated a few developers to this task.
	277	+senior developers were crucial at this point. They came up with an initial
	278	+solution to get us started. That already enabled us to automatize the deploy,
	279	+even though the process was still rudimentary. We had to evolve our solution
	280	+on-the-fly. We dedicated a few developers to this task.
286	281
287		-Building a CD pipeline was hard in the beginning, and we succeed thanks to the
288		-seniors experience. We believe that more tools that provide out-of-the-box
289		-standardized CD pipelines would be of great help for inexperienced teams. Tools
290		-that track each step of the pipeline and organize logs in a human-manageable
291		-way are necessary too.
	282	+Building a CD pipeline was hard in the beginning. We believe that more tools
	283	+that provide out-of-the-box standardized CD pipelines would be of great help
	284	+for inexperienced teams. Tools that track each step of the pipeline and
	285	+organize logs in a human-manageable way are necessary too.
292	286
293	287	### Handling inexperienced teams
294	288
...	...	@@ -303,14 +297,12 @@ are needed.
303	297	In the project’s first half we struggled with deploy related problems in the
304	298	government structure. We were in a paradoxical situation. The government
305	299	demanded speedy deliveries but would not give access to their production
306		-infrastructure. As an example, only in specific situations the government
307		-allowed us to access the PE. It took some interactions with government agents
308		-before we convinced them to create the VE as an isolated replica of the PE in
309		-their own infrastructure. The government agents then realized that it could be
310		-beneficial for the project if they granted us access to part of the structure.
311		-We believe it is required more research on development protocols and policies
312		-to improve the relation between industry and government, specially regarding
313		-CD.
	300	+infrastructure. Afterwards some interactions with government agents, they
	301	+created the VE as an isolated replica of the PE in their own infrastructure.
	302	+The government agents then realised that it could be beneficial for the project
	303	+if they granted us access to part of the structure. We believe it is required
	304	+more research on development protocols and policies to improve the relation
	305	+between industry and government, specially regarding CD.
314	306
315	307	## References
316	308
...	...