Fix pipeline

Rodrigo Siqueira de Melo
1 parent 6cc43c1e
Showing 1 changed file with 37 additions and 36 deletions Show diff stats
ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md
@@ -96,7 +96,7 @@ worst political crisis after the re-democratization in Brazil.
  
 ![Deployment Pipeline](figures/pipeCD.png)
  
-The Figure 1 represents our CD pipeline. The pipeline started when new code arrived.
+Figure 1 represents our CD pipeline. The pipeline started when new code arrived.
 As it went through each step, it was tested and improved until it finally reached
 the production environment. At this point we got back to the first stage to release
 more new code.
@@ -105,26 +105,26 @@ more new code.
  
 Each one of the Portal's software components had their own test suite. The
 development teams were required to maintain and expand them if possible. The
-communication of all these software was based on a plugin architecture. Testing
-the plugins were our integration tests. Finally, all tests ran in each
-component continuous integration environment.
+communication of all these software is orchestrated by Colab, which is based on
+a plugin architecture. Every component integrated into the Portal had a
+specific plugin developed by us. For instance, we developed integration
+plugins for Gitlab and Mailman. Plugins had their own test suite that
+served as integration tests.
  
-This two-level testing provided us with the necessary speed and trust that
-guaranteed both component and platform stability. If any error was found, the
-pipeline stopped and the developers were notified. Only after all tests passed
-we moved to preparing the release.
+Both the unit and integration tests provided us the necessary speed and trust
+that guaranteed both component and platform stability. If any test suite failed,
+either by a test error or coverage reduction below a certain threshold, the
+pipeline stopped. Only after all tests passed we moved to prepare the release.
  
 ### Preparing a new release
  
-Our release process was divided in two perspectives in terms of Git tags: the
-application and the SPB Portal. The application tag referred to the specific
-feature or bug fix and was a monotonically increasing. A new tag on any system
-yielded a new SPB Portal tag.
-
-When all tests passed for a given component, we manually created a new
-application tag for it. As a consequence, that automatically created a new tag
-for the SPB Portal. Notice that we forked the original software projects
-and, as consequence, we had different tag values.
+A SPB Portal release was composed of all its software components releases. Each
+software component release was a git tag that referred to a specific feature or
+bug fix. When all tests passed for a given component, we manually created a new
+tag for it. Therefore, a new tag on any software component yielded a new SPB
+Portal release. More precisely, SPB had a script that produced a single release
+for the entire system based on each component tag. At the end of this process,
+we started packaging.
  
 ### Packaging
  
@@ -133,39 +133,40 @@ packaging a software for that distribution has three steps: write the script
 for the specific environment (RPM), build the package, and upload it to a
 package repository.
  
-We chose to create our own packages for each software component for several
+We chose to create our own packages for each software component for five
 reasons:
  
 * Not all software was packaged by the community and those that existed were
 outdated;
 * Packaging makes it easy to manage the software on a given distribution;
-* It simplifies the deployment;
+* Packaging simplifies the deployment;
 * Packaging follows the distribution's best practices and,
-* Allows configurations and permissions control.
+* Packaging allows configurations and permissions control.
  
-After creating a new tag for one component, the DevOps team was notified and
-the packaging process began. The three packaging steps aforementioned were fully
-automated by a set of scripts. With all these scripts running successfully,
-the new packages would be ready to be used by our subsequent deployment scripts.
+After creating a new tag for one component, the developers informed the
+DevOps [3] team and the packaging process began. The three packaging steps
+aforementioned were fully automated by a set of scripts. With all these scripts
+running successfully, the new packages would be ready to be used by our
+subsequent deployment scripts.
  
 ### Validation Environment Deployment
  
 The Validation Environment (VE) is a replica of the Production Environment
-(PE), with two exceptions: only the government officers and us had access to it
-and all the data is anonymised. To configure the environment, we used
-our configuration management tool: Chake (serverless configuration with Chef).
-That maintained environment consistency simplifying the deployment process.
-Additionally, the packages we built on the last step were readily available to be
-used by the management tool.
-
-The VE was used by the government agents to validate new features and required
+(PE), with two exceptions: only the government officers and project leaders had
+access to it and all the data was anonymised. To configure the environment, we
+used a configuration management tool named Chef with Chake support (serverless
+configuration for Chef). That tool maintained environment consistency
+simplifying the deployment process.  Additionally, the packages we built on the
+last step were readily available to be used by the management tool.
+
+Government agents used the VE to validate new features and required
 changes. Also, the VE was useful to verify the integrity of the entire portal
 as part of the next step in the pipeline.
  
 ### Acceptance Tests
  
 After we deployed a new SPB Portal version in the VE, the government
-agents are responsible for checking features and bug fixes required by them. If
+agents were responsible for checking features and bug fixes required by them. If
 the requirement analysts identified a problem, they notified the developers via
 comments on the SPB Portal's issue tracker. The problems were fixed and the
 pipeline restarted from scratch. If everything was validated, we moved forward.
@@ -173,7 +174,7 @@ pipeline restarted from scratch. If everything was validated, we moved forward.
 ### Production Environment Deployment
  
 After the government finished the VE check and it was cleared for deployment,
-we could finally begin the deployment to Production Environment (PE). For this
+we could finally begin the deployment to the Production Environment (PE). For this
 we also used our configuration management tool, the same scripts and package
 versions as in the VE. After the deploy was completed, both VE and PE
 were running identical software. This was the point where new features and bug
@@ -297,6 +298,6 @@ between industry and government, specially regarding CD.
 ## References
  
 1. G. Booch, A. W. Brown, "Collaborative Development Environments", in Advances in Computers, vol. 59, 2003, pp. 1–27.
-1. L. Chen, "Continuous Delivery: Huge Benefits, but Challenges Too", in IEEE Software, vol. 32, no. 2, 2015, pp. 50-54.
-
+2. L. Chen, "Continuous Delivery: Huge Benefits, but Challenges Too", in IEEE Software, vol. 32, no. 2, 2015, pp. 50-54.
+3. Davis, Jennifer and Daniels, Katherine, Effective DevOps: building a culture of collaboration, affinity, and tooling at scale, 2016, " O'Reilly Media, Inc."
...	...	@@ -96,7 +96,7 @@ worst political crisis after the re-democratization in Brazil.
96	96
97	97	![Deployment Pipeline](figures/pipeCD.png)
98	98
99		-The Figure 1 represents our CD pipeline. The pipeline started when new code arrived.
	99	+Figure 1 represents our CD pipeline. The pipeline started when new code arrived.
100	100	As it went through each step, it was tested and improved until it finally reached
101	101	the production environment. At this point we got back to the first stage to release
102	102	more new code.
...	...	@@ -105,26 +105,26 @@ more new code.
105	105
106	106	Each one of the Portal's software components had their own test suite. The
107	107	development teams were required to maintain and expand them if possible. The
108		-communication of all these software was based on a plugin architecture. Testing
109		-the plugins were our integration tests. Finally, all tests ran in each
110		-component continuous integration environment.
	108	+communication of all these software is orchestrated by Colab, which is based on
	109	+a plugin architecture. Every component integrated into the Portal had a
	110	+specific plugin developed by us. For instance, we developed integration
	111	+plugins for Gitlab and Mailman. Plugins had their own test suite that
	112	+served as integration tests.
111	113
112		-This two-level testing provided us with the necessary speed and trust that
113		-guaranteed both component and platform stability. If any error was found, the
114		-pipeline stopped and the developers were notified. Only after all tests passed
115		-we moved to preparing the release.
	114	+Both the unit and integration tests provided us the necessary speed and trust
	115	+that guaranteed both component and platform stability. If any test suite failed,
	116	+either by a test error or coverage reduction below a certain threshold, the
	117	+pipeline stopped. Only after all tests passed we moved to prepare the release.
116	118
117	119	### Preparing a new release
118	120
119		-Our release process was divided in two perspectives in terms of Git tags: the
120		-application and the SPB Portal. The application tag referred to the specific
121		-feature or bug fix and was a monotonically increasing. A new tag on any system
122		-yielded a new SPB Portal tag.
123		-
124		-When all tests passed for a given component, we manually created a new
125		-application tag for it. As a consequence, that automatically created a new tag
126		-for the SPB Portal. Notice that we forked the original software projects
127		-and, as consequence, we had different tag values.
	121	+A SPB Portal release was composed of all its software components releases. Each
	122	+software component release was a git tag that referred to a specific feature or
	123	+bug fix. When all tests passed for a given component, we manually created a new
	124	+tag for it. Therefore, a new tag on any software component yielded a new SPB
	125	+Portal release. More precisely, SPB had a script that produced a single release
	126	+for the entire system based on each component tag. At the end of this process,
	127	+we started packaging.
128	128
129	129	### Packaging
130	130
...	...	@@ -133,39 +133,40 @@ packaging a software for that distribution has three steps: write the script
133	133	for the specific environment (RPM), build the package, and upload it to a
134	134	package repository.
135	135
136		-We chose to create our own packages for each software component for several
	136	+We chose to create our own packages for each software component for five
137	137	reasons:
138	138
139	139	* Not all software was packaged by the community and those that existed were
140	140	outdated;
141	141	* Packaging makes it easy to manage the software on a given distribution;
142		-* It simplifies the deployment;
	142	+* Packaging simplifies the deployment;
143	143	* Packaging follows the distribution's best practices and,
144		-* Allows configurations and permissions control.
	144	+* Packaging allows configurations and permissions control.
145	145
146		-After creating a new tag for one component, the DevOps team was notified and
147		-the packaging process began. The three packaging steps aforementioned were fully
148		-automated by a set of scripts. With all these scripts running successfully,
149		-the new packages would be ready to be used by our subsequent deployment scripts.
	146	+After creating a new tag for one component, the developers informed the
	147	+DevOps [3] team and the packaging process began. The three packaging steps
	148	+aforementioned were fully automated by a set of scripts. With all these scripts
	149	+running successfully, the new packages would be ready to be used by our
	150	+subsequent deployment scripts.
150	151
151	152	### Validation Environment Deployment
152	153
153	154	The Validation Environment (VE) is a replica of the Production Environment
154		-(PE), with two exceptions: only the government officers and us had access to it
155		-and all the data is anonymised. To configure the environment, we used
156		-our configuration management tool: Chake (serverless configuration with Chef).
157		-That maintained environment consistency simplifying the deployment process.
158		-Additionally, the packages we built on the last step were readily available to be
159		-used by the management tool.
160		-
161		-The VE was used by the government agents to validate new features and required
	155	+(PE), with two exceptions: only the government officers and project leaders had
	156	+access to it and all the data was anonymised. To configure the environment, we
	157	+used a configuration management tool named Chef with Chake support (serverless
	158	+configuration for Chef). That tool maintained environment consistency
	159	+simplifying the deployment process. Additionally, the packages we built on the
	160	+last step were readily available to be used by the management tool.
	161	+
	162	+Government agents used the VE to validate new features and required
162	163	changes. Also, the VE was useful to verify the integrity of the entire portal
163	164	as part of the next step in the pipeline.
164	165
165	166	### Acceptance Tests
166	167
167	168	After we deployed a new SPB Portal version in the VE, the government
168		-agents are responsible for checking features and bug fixes required by them. If
	169	+agents were responsible for checking features and bug fixes required by them. If
169	170	the requirement analysts identified a problem, they notified the developers via
170	171	comments on the SPB Portal's issue tracker. The problems were fixed and the
171	172	pipeline restarted from scratch. If everything was validated, we moved forward.
...	...	@@ -173,7 +174,7 @@ pipeline restarted from scratch. If everything was validated, we moved forward.
173	174	### Production Environment Deployment
174	175
175	176	After the government finished the VE check and it was cleared for deployment,
176		-we could finally begin the deployment to Production Environment (PE). For this
	177	+we could finally begin the deployment to the Production Environment (PE). For this
177	178	we also used our configuration management tool, the same scripts and package
178	179	versions as in the VE. After the deploy was completed, both VE and PE
179	180	were running identical software. This was the point where new features and bug
...	...	@@ -297,6 +298,6 @@ between industry and government, specially regarding CD.
297	298	## References
298	299
299	300	1. G. Booch, A. W. Brown, "Collaborative Development Environments", in Advances in Computers, vol. 59, 2003, pp. 1–27.
300		-1. L. Chen, "Continuous Delivery: Huge Benefits, but Challenges Too", in IEEE Software, vol. 32, no. 2, 2015, pp. 50-54.
301		-
	301	+2. L. Chen, "Continuous Delivery: Huge Benefits, but Challenges Too", in IEEE Software, vol. 32, no. 2, 2015, pp. 50-54.
	302	+3. Davis, Jennifer and Daniels, Katherine, Effective DevOps: building a culture of collaboration, affinity, and tooling at scale, 2016, " O'Reilly Media, Inc."
302	303
...	...