diff --git a/ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md b/ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md index 459978e..87d2f55 100644 --- a/ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md +++ b/ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md @@ -1,5 +1,5 @@ --- -title: "Continuous Delivery: Building Trust in a Large-scale, complex government organization" +title: "Continuous Delivery: Building Trust in a Large-scale, Complex Government Organization" papersize: a4 geometry: "left=1in,right=1.5in" --- @@ -133,20 +133,19 @@ project, we became active participants in the deploy operations. SPB portal is a CDE with additional social features. We built system-of-systems adapting and integrating five existing OSS projects: Colab (www.github.com/colab), Noosfero (www.noosfero.org), Gitlab (www.gitlab.com), -Mezuro (www.mezuro.org), and Mailman (www.list.org). Colab orchestrates these +Mezuro (www.mezuro.org), and Mailman (www.list.org). Colab orchestrates these multiple components and allowed us to smoothly provide a unified interface for -final users, including single sign-on and global searches [1]. All integrated +final users, including single sign-on and global searches [1]. All these integrated system represents a total of 106.253 commits and 1.347.421 lines of code. ![The SPB Deployment Pipeline.](figures/pipeline.png) -The SPB portal deployment follows a typical CD pipeline [3], adapted to the +The portal deployment follows a typical CD pipeline [3], adapted to the technical and organizational context of our project and the use of OSS best practices, as presented in Figure 1. It started when new code arrived; and when it reach the production environment, we would restart the pipeline to release new versions. - ### Automated Tests Each integrated system had to be tested with its own test suite. This checking @@ -159,10 +158,10 @@ step of release preparation. ### Preparing a New Release -Each software component was hosted in a separate Git repository. A new release -of a component was tagged referencing a specific new feature or bug fix. SPB +Each software component was hosted in a separate Git repository. A new component +release was tagged referencing a specific new feature or bug fix. SPB had its own Git repository (www.softwarepublico.gov.br/gitlab/softwarepublico). -An SPB portal release was an aggregate of versions of all of its components. +An SPB portal release was an aggregate of versions of all its components. When a new component release passed all of the SPB integration tests, we manually created a corresponding new tag in its repository. Therefore, a new tag on any software component yielded a new SPB portal release. At the end of @@ -170,37 +169,32 @@ this process, we started packaging. ### Packaging -Our packaging software approach involves three steps: writing the script for -the specific environment, building the package, and uploading it to a package -repository. We decided to create separate packages for each system. Packaging -brings portability, simplifies deployment, and enables configuration and -permission control. - -After creating a new tag for a system, the developers informed our DevOps -team, and the packaging process began. A set of scripts fully automated the -three packaging steps aforementioned. When all of them ran successfully, the -new packages would be ready and available for our deployment scripts. +After creating a new tag for a system, DevOps team starts the packaging process. +Packaging brings portability, simplifies deployment, and enables configuration +and permission control. Our packaging software approach involves building +separate packages for each system in three steps: writing the script for the +specific environment, building the package, and uploading it to a package +repository. A set of scripts fully automated these steps. When all of them ran +successfully, the new packages would be ready and available for our deployment +scripts. ### Validation Environment Deployment The Validation Environment (VE) is a replica of the Production Environment (PE) with anonymised data, and access restricted to Ministry staff and our DevOps -team. To configure this environment, we used Chef (www.chef.io) and Chake, a -serverless configuration tool created by our team +team. To configure this environment, we used Chef (www.chef.io) and Chake, a +serverless configuration tool created by our team (www.github.com/terceiro/chake). This tool maintained environment consistency, simplifying the deployment process. -The Ministry staff used the VE to validate new features and required changes. -The VE also was used to verify the integrity of the entire portal as part of -the next step in the pipeline. - ### Acceptance Tests -After a new SPB portal deployment in the VE, the Ministry was responsible for -checking the required features and bug fixes. If they identified a problem, -they would notify the developers via comments on the SPB portal's issue -tracker, prompting the team to fix it and restart the pipeline. Otherwise, we -could move forward. +After a new SPB portal deployment in the VE, we used the environment to verify +the integrity of the entire portal. In this step, Ministry staff also checked +the new features, required changes and bug fixes. If they identified a problem, +they would notify the developers via comments on the SPB portal's issue tracker, +prompting the team to fix it and restart the pipeline. Otherwise, we could move +forward. ### Production Environment Deployment diff --git a/ieeeSW/releaseEng3/figures/CD-Evolution.svg b/ieeeSW/releaseEng3/figures/CD-Evolution.svg deleted file mode 100644 index d4c2135..0000000 --- a/ieeeSW/releaseEng3/figures/CD-Evolution.svg +++ /dev/null @@ -1,1033 +0,0 @@ - - - - - - image/svg+xml - - - - - - Created with Highcharts 6.0.3 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Date - - - - - - - - - - - - Release Evolution - - - From Jan/2014 to Mai/2016 - - - - - - - Release # - - - - Amount of releases - - - - - Jul '14 - - - Jan '15 - - - Jul '15 - - - Jan '16 - - - Jul '16 - - - - 0 - - - 25 - - - 50 - - - 75 - - - 100 - - - - - - - - - - - - - - - - - - - - - - - - - - - Date - - - - - - - - - - - - - - - - - - - Team Members Distribution - - - From Jan/2014 to Mai/2016 - - - - - - # Members Development Team - - - - - Qt of DevOps Members - - - - Jul '14 - - - Jan '15 - - - Jul '15 - - - Jan '16 - - - Jul '16 - - - - 0 - - - 10 - - - 20 - - - 30 - - - 40 - - - 50 - - - 27 - - - 46 - - - 23 - - - 6 - - - 0 - - - 6 - - - 9 - - - 11 - - - 6 - - - - - - 5 - - - 15 - - - 30 - - - 57 - - - 84 - - - - diff --git a/ieeeSW/releaseEng3/figures/pipeline.svg b/ieeeSW/releaseEng3/figures/pipeline.svg deleted file mode 100644 index fb46fd5..0000000 --- a/ieeeSW/releaseEng3/figures/pipeline.svg +++ /dev/null @@ -1,1691 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - image/svg+xml - - - - - - - - - Manually - - Automated - - Packaging - - - - SPB release - - ValidationEnvironmentDeployment - - - Newfeatureavailable - - - AcceptanceTests - - - WaitforAnalystsfeedback - ProductionEnvironment - - - Newfeatureavailable - Automatedtesting - - - - - System 1unit tests - - - System 2unit tests - - System 3unit tests - - System 4unit tests - - Plugin tests - ... - - - - - Colab - - - - - Colab - - ImplementedCode - ... ... - System 2 - - System 3 - - System 4 - - - System 1 - - - Feature - Preparingnew Release - - ... - System 1release - - System 2release - - System 3release - - System 4release - - - - - - - - - - - - - - - - - - - - - - - - - PackageSet - - diff --git a/ieeeSW/releaseEng3/figures/raw/CD-Evolution.svg b/ieeeSW/releaseEng3/figures/raw/CD-Evolution.svg new file mode 100644 index 0000000..d4c2135 --- /dev/null +++ b/ieeeSW/releaseEng3/figures/raw/CD-Evolution.svg @@ -0,0 +1,1033 @@ + + + + + + image/svg+xml + + + + + + Created with Highcharts 6.0.3 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Date + + + + + + + + + + + + Release Evolution + + + From Jan/2014 to Mai/2016 + + + + + + + Release # + + + + Amount of releases + + + + + Jul '14 + + + Jan '15 + + + Jul '15 + + + Jan '16 + + + Jul '16 + + + + 0 + + + 25 + + + 50 + + + 75 + + + 100 + + + + + + + + + + + + + + + + + + + + + + + + + + + Date + + + + + + + + + + + + + + + + + + + Team Members Distribution + + + From Jan/2014 to Mai/2016 + + + + + + # Members Development Team + + + + + Qt of DevOps Members + + + + Jul '14 + + + Jan '15 + + + Jul '15 + + + Jan '16 + + + Jul '16 + + + + 0 + + + 10 + + + 20 + + + 30 + + + 40 + + + 50 + + + 27 + + + 46 + + + 23 + + + 6 + + + 0 + + + 6 + + + 9 + + + 11 + + + 6 + + + + + + 5 + + + 15 + + + 30 + + + 57 + + + 84 + + + + diff --git a/ieeeSW/releaseEng3/figures/raw/pipeline.svg b/ieeeSW/releaseEng3/figures/raw/pipeline.svg new file mode 100644 index 0000000..fb46fd5 --- /dev/null +++ b/ieeeSW/releaseEng3/figures/raw/pipeline.svg @@ -0,0 +1,1691 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + Manually + + Automated + + Packaging + + + + SPB release + + ValidationEnvironmentDeployment + + + Newfeatureavailable + + + AcceptanceTests + + + WaitforAnalystsfeedback + ProductionEnvironment + + + Newfeatureavailable + Automatedtesting + + + + + System 1unit tests + + + System 2unit tests + + System 3unit tests + + System 4unit tests + + Plugin tests + ... + + + + + Colab + + + + + Colab + + ImplementedCode + ... ... + System 2 + + System 3 + + System 4 + + + System 1 + + + Feature + Preparingnew Release + + ... + System 1release + + System 2release + + System 3release + + System 4release + + + + + + + + + + + + + + + + + + + + + + + + + PackageSet + + -- libgit2 0.21.2