From 2b94e087a4fca0ee5f0c2709e9cf05025e8dc9c5 Mon Sep 17 00:00:00 2001 From: Rafael Reggiani Manzo Date: Thu, 27 Jul 2017 19:37:09 -0300 Subject: [PATCH] Create pipeline image --- ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md | 561 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ieeeSW/releaseEng3/figures/pipeline.dia | Bin 0 -> 2167 bytes ieeeSW/releaseEng3/figures/pipeline.png | Bin 0 -> 64981 bytes 3 files changed, 283 insertions(+), 278 deletions(-) create mode 100644 ieeeSW/releaseEng3/figures/pipeline.dia create mode 100644 ieeeSW/releaseEng3/figures/pipeline.png diff --git a/ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md b/ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md index ff47f7f..27b4275 100644 --- a/ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md +++ b/ieeeSW/releaseEng3/IEEE_ThemeIssue_ReleaseEng_CD.md @@ -1,278 +1,283 @@ -# The Strategic Importance of Continuous Delivery - -## Abstract - -Many software development teams think of the operational aspects of Continuous -Delivery (CD) and the competitive benefits that come along with it. For us, it -was much more: it was a survival technique. This article presents the -experience applying CD in a Brazilian Government project for the development of -a Collaborative Development Environment (CDE), sharing its unconventional -challenges and our strategies used to overcome them. This report from the -trenches of the Brazilian Federal Government can help practitioners to -understand how important the CD adoption is to their projects. - -## Introduction and Context - -We have worked on a Brazilian government three-year-long project to redesign an -existing platform that had technical issues and a lack of political support. -This evolution project started in a presidential election year and everyone -involved were under Presidential re-election campaign pressure to show results. -Even with the re-election of the Brazilian President, leadership in government -agencies suddenly changed that reflected on the project’s requirements: each -new leader wanted to fulfill their political agenda. In this scenario, delivery -delays could have sunk the project into oblivion. - -From 2013 to 2016, our team developed the new platform for the Brazilian Public -Software (SPB, Portuguese acronym) Portal (www.softwarepublico.gov.br). The SPB -Portal has evolved to a CDE [1] and this evolution has brought important -benefits not just to the Brazilian government, but also to society as a whole. -For the government, the bureaucracy on using the same software across -government agencies, duplicate works and costs all are reduced. The society -gains a transparency and collaboration mechanism, since anyone can check the -government expenses on software and contribute to the software communities. To -achieve these goals, rather than writing everything from scratch, we have -chosen to integrate free software tools such as Gitlab (www.gitlab.com), -Mailman (www.gnu.org/software/mailman), Noosfero (www.noosfero.org), and -Colab (www.github.com/colab). - -During the entire SPB Portal evolution project, we had to handle three distinct -issues, usual in a software engineering scenario: reaching the goals which have -guided the platform development, managing the diversity of team project -members, and communicating effectively with clients (in this particular case, -government agents). Managing the interaction of these elements was not easy and -the unstable Brazilian political scenario only made things worse. - -To reach the SPB project goals, we had to overcome strong political bias -tied with complicated technical issues and relatively low budget. Because it is -open to the public, the government representatives have seen the platform as a -marketing opportunity and have often ignored the technical advice in favor -of political decisions. Furthermore, integrating a number of distinct systems -to work seamlessly was not an easy job. We had to learn how each system worked -and come up with ideas of how to integrate them as fast as possible, with a -team of mostly inexperienced developers. - -We also had to manage the diversity of the SPB team members. This team was -composed of approximately 50 undergraduate students (not all simultaneously) -together with professors, masters students and professional designers as well -as senior developers from the Brazilian free software community. Undergraduate -students have received a fellowship and, for most of them, this R&D project was -their first professional experience. Seniors developers and masters students -had two important contributions to the project: transfer knowledge to -undergraduate students and address hard tasks. Finally, professors were -responsible for interacting with the Brazilian government and control the -political pressures applied to the project. - -Our third point to be handled was the communication with the two independent -groups of government representatives: requirements analysts and deployment -technicians. Requirements analysts were the real representatives of the -Brazilian government in the project. They usually tested new features, provided -feedback, and reported for directors. The deployment technicians only had the -access to the host machines wherein SPB platform was running. They were, -theoretically, responsible for deploying the project. However, the new SPB -Portal was composited from more than ten different software projects, -generating a complex deploy process working on seven servers. - -Therefore, we realised that we needed to take control over the deployment -process. We would use CD as a mean to keep the government satisfied and provide -quick response times to their requests which were, most of the time, influenced -by the uncertainties of project continuity. We believed that would keep the -project alive even in an politically unstable and complex technical scenario. -For that, during months we have worked to automate all the deploy process. For -instance, we have created a tool called Shak (Self Hosting Applications Kit) -that offers an application installation with only instruction. It aims to make -the deploy process easier for users without technical knowledge, ensuring -privacy and security for their Internet applications. Thereby, we created a -specific team dedicated to the deployment process to mature our CD pipeline -that would give us confidence to meet the government’s requirements faster and -faster. - -We describe our CD pipeline and how it speeded up our software delivery time. -The CD made us adaptable for all requested changes and helped us to mitigate -those aforementioned political challenges besides the technical issues. Among -CD’s known benefits[2], we also explain those that were the most important in -the project scenario for us: improving customer satisfaction and making -reliable releases. Both kept the project alive for more two years during the -worst political crisis after the re-democratization in Brazil. - -## Our Continuous Delivery Pipeline - -[//]: # (TODO - Fazer a imagem. Sem esquentar muito a cabeça, uma vez que a revista deve refazer tal imagem) - -[//]: # (conjunto de features -> testes automatizados → criação de tag → empacotamento →deploy em homologação → testes de aceitação → deploy em produção) - -[//]: # (TODO - Rever essa frase - é uma legenda da imagem?) - -Figure X represents our CD pipeline. After each release, we have defined a set -of features we would develop to comply with government’s requirements and then -we have followed this pipeline. - -### Automated tests - -[//]: # (TODO - Rever a escrita desta ideia com mais calma) -[//]: # (TODO - Adicionar que as mudanças no Colab quebravam todos os plugins.) -[//]: # (https://www.linux.com/blog/learn/chapter/dev-ops/2017/7/devops-fundamentals-part-3-continuous-delivery-and-deployment) - -The SPB portal is composed of a set of FOSS systems. All of them have their own -automated test suite. So, we encouraged the teams to keep unit tests coverage -high for each system. We implemented the systems integration using a plugin -architecture and wrote integration tests to check if they were working -properly. The plugin architecture made the PSPB’s components decoupled. That -allowed us to be confident that changes on one system would not affect others. -This characteristic allowed the test execution of only the component with -changes instead of the entire system. - -Our CD’s first step was to execute each system’s automated test suite. If any -error was found, the pipeline stopped and the developers were notified. Only -after all tests passed we move to preparing the release. Preparing a new -release - -Our release system was divided into two perspectives: the application and the -PSPB. The application tag refers to the specific feature or bug fix and is a -monotonically increasing. A new tag on any system yielded a new PSPB tag. - -When all tests passed for a given component, we manually created a new -application tag for it. As a consequence, that automatically created a new tag -for the PSPB. Notice that we have forks of the original softwares and, as -consequence, we had different tag values. Packaging - -The PSPB platform is running under the CentOS GNU/Linux distribution. -Basically, packaging a software for that distribution has three steps: (1) -write the script for the specific environment (RPM), (2) build the package, and -(3) upload it to a package repository. We chose to package our components for -several reasons: -* Not all software was packaged by the community; -* And those that existed were outdated; -* Packaging makes it easy to manage the software on a given distribution; -* It simplifies the deployment; -* Packaging follows the distribution’s best practices and, -* Allows configurations and permissions control. - -After creating a new tag for one component, the DevOps team was notified and -packaging process began. In the normal case, the three packaging steps -aforementioned are fully automated by a set of scripts. However, if the team -reports to DevOps any eventual dependency change, the first step has to be -manual. For instance, one system could start requiring another system to be -initialized and that made necessary for someone to manually update the package -script. - -After all these scripts have run successfully, the new packages would be ready -to use by our subsequent deployment scripts. - -### Validation Environment - -[//]: # (TODO - Mencionar que a ferramenta era baseada em Chef - Dá um peso importante.) - -The Validation Environment (VE) is a replica of the Production Environment -(PE), with two exceptions: only the government officers and us had access to it -and all the data is anonymised. To configure the environment, we use a -configuration management tool. That maintained environment consistency which -makes the deployment process simpler. Additionally, the packages we built on -the last step were readily available to use by the management tool. - -The VE was used by the government agents to validate new features and required -changes. Also, the VE was useful to verify the integrity of the entire portal -as part of the next step in the pipeline. Acceptance Tests - -After we completely deploy a new PSPB version in the VE, the government agents -are responsible for checking features and/or bug fixes required by them. If the -technicians identify a problem, they notify the developers, the problems are -fixed and the pipeline restarts from scratch. If everything is validated, we -move to production deployment. Production Deployment - -After the government authorizes the VE, we can finally begin the production -deployment. We use the same configuration management tool with the same -scripts. After the deploy is completed, both VE and PE are on the same state. -The new features and bug fixes are finally available to end users. - - -## Benefits - -We had to handle many tensions between development and political issues. Our CD -pipeline gave us strong mechanisms to tackle most of the problems. As a result -we came with some benefits from our decision to adopt CD. - -[//]: # (TODO - Melhorar título - Ideias: Response to mistrust) - -### Response to tensions - -[//]: # (TODO - Decisão do secretario acima do diretor) - -The direct benefit from the CD pipeline was the fast response to the changes -required by the government. That was vital for the project’s renewal over the -years. We could manage the tension between the government and the development -team better. Every meeting with the government leader was delicate and resulted -on many new requirements, most of them motivated by political needs. For -example, once it was demanded a completely layout change because one director -suddenly decided to make a marketing campaign about the portal. They would use -undelivered requirements as a means to suggest the project’s cancellation. We -believed that if we took too long to attend their demands, the project would -end. CD helped us to move fast on deploying to production, even of smaller -parts of the requirements. That way, we always had something to show on the -meetings, reducing their eagerness to end the project. For our team, it made -the developers more confident the project would last a little longer and they -would not go looking for another jobs. - -### Build client’s trust - -After we established the CD, the government agents started to be more confident -in our work. First, because they noticed that each new deploy made by us in the -VE was stable and reliable. Second, they could see new features fast since we -constantly updated the VE based on their feedback. This made our relation -strong and in moments that needed quick action they would rather give us access -to production. - - -## Challenges - -We successfully built a functional CD pipeline. In the end, we took over the -deployment process from the government. That allowed us to survive into an -unstable political scenario. However, we recognized that many challenges still -need to be addressed by the industry and academia together. Build CD from -scratch - -Taking on CD responsibilities had a significant impact on the team. We did not -have the know-how and had little time to come up with a working pipeline. To -make things worse, we were not aware of how companies normally organized their -teams to make CD feasible. - -[//]: # (TODO - Deixar uma abertura para uma eventual pesquisa) - -The seniors were crucial at this point. They came up with an initial solution -to get us started. That already enabled us to automatize the deploy, even -though the process was still rudimentary. We had to evolve our solution -on-the-fly. We dedicated a few developers to this task. - -### Handling inexperienced teams - -After the developers learned how CD worked, it was difficult to pass the -knowledge along to other teammates. We tried to mitigate this by encouraging a -member's migration to the DevOps team. Further research on how to effectively -spread knowledge across inexperienced developers in a scenario with a high -turnover are needed. - -[//]: # (TODO - Refazer título abaixo - ideias: overcoming the challenges / overcoming the mistrust) - -### Building trust - -[//]: # (TODO - Deixar claro que government = os técnicos , análista ou agentes do governo, conforme for padronizado no texto) - -In the project’s first half we struggled with deploy related problems in the -government structure. We were in a paradoxical situation. The government -demanded speedy deliveries but would not give access to their production -infrastructure. As an example, only in a very specific situation the government -allowed us to access the PE. After some interactions with the government we -convinced them to create the VE as an isolated replica of the PE in their own -infrastructure. The government agents then realized that it could be good for -the project if they granted us access to part of the structure since we could -deliver new features to them faster. We believe it is required more research on -development protocols and policies to improve the relation between industry and -government, specially regarding CD. - - -## References - -1. G. Booch, A. W. Brown, "Collaborative Development Environments, in Advances in Computers", vol. 59, 2003, pp. 1–27. -1. L. Chen, "Continuous Delivery: Huge Benefits, but Challenges Too", in IEEE Software, vol. 32, no. 2, 2015, pp. 50-54. -1. P. Meirelles, M. Wen, A. Terceiro, R. Siqueira, L. Kanashiro, and H. Neri, "Brazilian Public Software Portal: an integrated platform for collaborative development", in Proceedings of the 13th International Symposium on Open Collaboration, 2017. - - +# The Strategic Importance of Continuous Delivery + +## Abstract + +Many software development teams think of the operational aspects of Continuous +Delivery (CD) and the competitive benefits that come along with it. For us, it +was much more: it was a survival technique. This article presents the +experience applying CD in a Brazilian Government project for the development of +a Collaborative Development Environment (CDE), sharing its unconventional +challenges and our strategies used to overcome them. This report from the +trenches of the Brazilian Federal Government can help practitioners to +understand how important the CD adoption is to their projects. + +## Introduction and Context + +We have worked on a Brazilian government three-year-long project to evolve an +existing platform that had technical issues and a lack of political support. +The evolution project started in a presidential election year and everyone +involved were under Presidential re-election campaign pressure to show results. +Even with the re-election of the Brazilian President in 2014, leadership in +government agencies suddenly changed, what was worsened by corruption scandals +revealed by the Brazilian Federal police, in particular the "Car-Wash" +investigation. That reflected on the project’s requirements: each new leader +wanted to fulfill their political agenda. In this scenario, delivery delays +could have sunk the project into oblivion. + +From 2014 to 2016, our team developed the new platform for the Brazilian Public +Software (SPB, Portuguese acronym) Portal (www.softwarepublico.gov.br) funded +by a grant of 2,619,965.00 BRL (about 1,000,000.00 USD in January 2014) from +the Federal Government. The SPB Portal has evolved to a CDE [1] and this +evolution has brought important benefits not just to the Brazilian government, +but also to society as a whole. For the government, the bureaucracy on using +the same software across government agencies, duplicate works and costs all are +reduced. The society gains a transparency and collaboration mechanism, since +anyone can check the government expenses on software and contribute to the +software communities. To achieve these goals, rather than writing everything +from scratch, we have chosen to integrate free software tools such as Gitlab +(www.gitlab.com), Mailman (www.gnu.org/software/mailman), Noosfero +(www.noosfero.org), and Colab (www.github.com/colab). + +During the entire SPB Portal evolution project, we had to handle three distinct +issues, usual in a software engineering scenario: reaching the goals which have +guided the platform development, managing the diversity of team project +members, and communicating effectively with clients (in this particular case, +government agents). Managing the interaction of these elements was not easy and +the unstable Brazilian political scenario only made things worse. + +To reach the SPB project goals, we had to overcome strong political bias +tied with complicated technical issues and relatively low budget. Because it is +open to the public, the government representatives have seen the platform as a +marketing opportunity and have often ignored the technical advice in favor +of political decisions. Furthermore, integrating a number of distinct systems +to work seamlessly was not an easy job. We had to learn how each system worked +and come up with ideas of how to integrate them as fast as possible, with a +team of mostly inexperienced developers. + +We also had to manage the diversity of the SPB team members. This team was +composed of approximately 50 undergraduate students (not all simultaneously) +together with professors, masters students and professional designers as well +as senior developers from the Brazilian free software community. Undergraduate +students have received a fellowship and, for most of them, this R&D project was +their first professional experience. Seniors developers and masters students +had two important contributions to the project: transfer knowledge to +undergraduate students and address hard tasks. Finally, professors were +responsible for interacting with the Brazilian government and control the +political pressures applied to the project. + +Our third point to be handled was the communication with the two independent +groups of government representatives: requirements analysts and deployment +technicians. Requirements analysts were the real representatives of the +Brazilian government in the project. They usually tested new features, provided +feedback, and reported for directors. The deployment technicians only had the +access to the host machines wherein SPB platform was running. They were, +theoretically, responsible for deploying the project. However, the new SPB +Portal was composited from more than ten different software projects, +generating a complex deploy process working on seven servers. + +Therefore, we realised that we needed to take control over the deployment +process. We would use CD as a mean to keep the government satisfied and provide +quick response times to their requests which were, most of the time, influenced +by the uncertainties of project continuity. We believed that would keep the +project alive even in a politically unstable and technically complex scenario. +For that, during months we have worked to automate all the deploy process. For +instance, we have created a tool based on Chef called Shak (Self Hosting +Applications Kit) that offers an application installation interface based only + on instruction. It aims to make the deployment process easier for users + without technical knowledge, ensuring privacy and security for their Internet + applications. Thereby, we created a specific team dedicated to the deployment + process to mature our CD pipeline that would give us confidence to meet the + government’s requirements faster and faster. + +We describe our CD pipeline and how it speeded up our software delivery time. +The CD made us adaptable for all requested changes and helped us to mitigate +those aforementioned political challenges besides the technical issues. Among +CD’s known benefits[2], we also explain those that were the most important in +the project scenario for us: improving customer satisfaction and making +reliable releases. Both kept the project alive for more two years during the +worst political crisis after the re-democratization in Brazil. + +## Our Continuous Delivery Pipeline + +[//]: # (TODO - Fazer a imagem. Sem esquentar muito a cabeça, uma vez que a revista deve refazer tal imagem) + +[//]: # (conjunto de features -> testes automatizados → criação de tag → empacotamento →deploy em homologação → testes de aceitação → deploy em produção) + +[//]: # (TODO - Rever essa frase - é uma legenda da imagem?) + +![Deployment Pipeline](figures/pipeline.png) + +Figure X represents our CD pipeline. After each release, we have defined a set +of features we would develop to comply with government’s requirements and then +we have followed this pipeline. + +### Automated tests + +[//]: # (TODO - Rever a escrita desta ideia com mais calma) +[//]: # (TODO - Adicionar que as mudanças no Colab quebravam todos os plugins.) +[//]: # (https://www.linux.com/blog/learn/chapter/dev-ops/2017/7/devops-fundamentals-part-3-continuous-delivery-and-deployment) + +The SPB portal is composed of a set of FOSS systems. All of them have their own +automated test suite. So, we encouraged the teams to keep unit tests coverage +high for each system. We implemented the systems integration using a plugin +architecture and wrote integration tests to check if they were working +properly. The plugin architecture made the PSPB’s components decoupled. That +allowed us to be confident that changes on one system would not affect others. +This characteristic allowed the test execution of only the component with +changes instead of the entire system. + +Our CD’s first step was to execute each system’s automated test suite. If any +error was found, the pipeline stopped and the developers were notified. Only +after all tests passed we move to preparing the release. Preparing a new +release + +Our release system was divided into two perspectives: the application and the +PSPB. The application tag refers to the specific feature or bug fix and is a +monotonically increasing. A new tag on any system yielded a new PSPB tag. + +When all tests passed for a given component, we manually created a new +application tag for it. As a consequence, that automatically created a new tag +for the PSPB. Notice that we have forks of the original softwares and, as +consequence, we had different tag values. Packaging + +The PSPB platform is running under the CentOS GNU/Linux distribution. +Basically, packaging a software for that distribution has three steps: (1) +write the script for the specific environment (RPM), (2) build the package, and +(3) upload it to a package repository. We chose to package our components for +several reasons: +* Not all software was packaged by the community; +* And those that existed were outdated; +* Packaging makes it easy to manage the software on a given distribution; +* It simplifies the deployment; +* Packaging follows the distribution’s best practices and, +* Allows configurations and permissions control. + +After creating a new tag for one component, the DevOps team was notified and +packaging process began. In the normal case, the three packaging steps +aforementioned are fully automated by a set of scripts. However, if the team +reports to DevOps any eventual dependency change, the first step has to be +manual. For instance, one system could start requiring another system to be +initialized and that made necessary for someone to manually update the package +script. + +After all these scripts have run successfully, the new packages would be ready +to use by our subsequent deployment scripts. + +### Validation Environment + +[//]: # (TODO - Mencionar que a ferramenta era baseada em Chef - Dá um peso importante.) + +The Validation Environment (VE) is a replica of the Production Environment +(PE), with two exceptions: only the government officers and us had access to it +and all the data is anonymised. To configure the environment, we use a +configuration management tool. That maintained environment consistency which +makes the deployment process simpler. Additionally, the packages we built on +the last step were readily available to use by the management tool. + +The VE was used by the government agents to validate new features and required +changes. Also, the VE was useful to verify the integrity of the entire portal +as part of the next step in the pipeline. Acceptance Tests + +After we completely deploy a new PSPB version in the VE, the government agents +are responsible for checking features and/or bug fixes required by them. If the +technicians identify a problem, they notify the developers, the problems are +fixed and the pipeline restarts from scratch. If everything is validated, we +move to production deployment. Production Deployment + +After the government authorizes the VE, we can finally begin the production +deployment. We use the same configuration management tool with the same +scripts. After the deploy is completed, both VE and PE are on the same state. +The new features and bug fixes are finally available to end users. + + +## Benefits + +We had to handle many tensions between development and political issues. Our CD +pipeline gave us strong mechanisms to tackle most of the problems. As a result +we came with some benefits from our decision to adopt CD. + +[//]: # (TODO - Melhorar título - Ideias: Response to mistrust) + +### Response to tensions + +[//]: # (TODO - Decisão do secretario acima do diretor) + +The direct benefit from the CD pipeline was the fast response to the changes +required by the government. That was vital for the project’s renewal over the +years. We could manage the tension between the government and the development +team better. Every meeting with the government leader was delicate and resulted +on many new requirements, most of them motivated by political needs. For +example, once it was demanded a completely layout change because one director +suddenly decided to make a marketing campaign about the portal. They would use +undelivered requirements as a means to suggest the project’s cancellation. We +believed that if we took too long to attend their demands, the project would +end. CD helped us to move fast on deploying to production, even of smaller +parts of the requirements. That way, we always had something to show on the +meetings, reducing their eagerness to end the project. For our team, it made +the developers more confident the project would last a little longer and they +would not go looking for another jobs. + +### Build client’s trust + +After we established the CD, the government agents started to be more confident +in our work. First, because they noticed that each new deploy made by us in the +VE was stable and reliable. Second, they could see new features fast since we +constantly updated the VE based on their feedback. This made our relation +strong and in moments that needed quick action they would rather give us access +to production. + + +## Challenges + +We successfully built a functional CD pipeline. In the end, we took over the +deployment process from the government. That allowed us to survive into an +unstable political scenario. However, we recognized that many challenges still +need to be addressed by the industry and academia together. Build CD from +scratch + +Taking on CD responsibilities had a significant impact on the team. We did not +have the know-how and had little time to come up with a working pipeline. To +make things worse, we were not aware of how companies normally organized their +teams to make CD feasible. + +[//]: # (TODO - Deixar uma abertura para uma eventual pesquisa) + +The seniors were crucial at this point. They came up with an initial solution +to get us started. That already enabled us to automatize the deploy, even +though the process was still rudimentary. We had to evolve our solution +on-the-fly. We dedicated a few developers to this task. + +### Handling inexperienced teams + +After the developers learned how CD worked, it was difficult to pass the +knowledge along to other teammates. We tried to mitigate this by encouraging a +member's migration to the DevOps team. Further research on how to effectively +spread knowledge across inexperienced developers in a scenario with a high +turnover are needed. + +[//]: # (TODO - Refazer título abaixo - ideias: overcoming the challenges / overcoming the mistrust) + +### Building trust + +[//]: # (TODO - Deixar claro que government = os técnicos , análista ou agentes do governo, conforme for padronizado no texto) + +In the project’s first half we struggled with deploy related problems in the +government structure. We were in a paradoxical situation. The government +demanded speedy deliveries but would not give access to their production +infrastructure. As an example, only in a very specific situation the government +allowed us to access the PE. After some interactions with the government we +convinced them to create the VE as an isolated replica of the PE in their own +infrastructure. The government agents then realized that it could be good for +the project if they granted us access to part of the structure since we could +deliver new features to them faster. We believe it is required more research on +development protocols and policies to improve the relation between industry and +government, specially regarding CD. + + +## References + +1. G. Booch, A. W. Brown, "Collaborative Development Environments, in Advances in Computers", vol. 59, 2003, pp. 1–27. +1. L. Chen, "Continuous Delivery: Huge Benefits, but Challenges Too", in IEEE Software, vol. 32, no. 2, 2015, pp. 50-54. +1. P. Meirelles, M. Wen, A. Terceiro, R. Siqueira, L. Kanashiro, and H. Neri, "Brazilian Public Software Portal: an integrated platform for collaborative development", in Proceedings of the 13th International Symposium on Open Collaboration, 2017. + + diff --git a/ieeeSW/releaseEng3/figures/pipeline.dia b/ieeeSW/releaseEng3/figures/pipeline.dia new file mode 100644 index 0000000..12ca3d2 Binary files /dev/null and b/ieeeSW/releaseEng3/figures/pipeline.dia differ diff --git a/ieeeSW/releaseEng3/figures/pipeline.png b/ieeeSW/releaseEng3/figures/pipeline.png new file mode 100644 index 0000000..5350e52 Binary files /dev/null and b/ieeeSW/releaseEng3/figures/pipeline.png differ -- libgit2 0.21.2