Commit db4810a248c190845821d098c7e891a7418dd038
1 parent
6dd0b976
Exists in
master
and in
3 other branches
[opensym] Reviewing the conclusion
Showing
5 changed files
with
63 additions
and
59 deletions
Show diff stats
opensym2017/content/01-introduction.tex
| @@ -41,7 +41,7 @@ Approximately five years later from the first problems, a new platform for the | @@ -41,7 +41,7 @@ Approximately five years later from the first problems, a new platform for the | ||
| 41 | SPB Portal was planned and funded. Between January 2014 and June 2016, the | 41 | SPB Portal was planned and funded. Between January 2014 and June 2016, the |
| 42 | University of Brasília (UnB) and the University of São Paulo (USP) in a | 42 | University of Brasília (UnB) and the University of São Paulo (USP) in a |
| 43 | partnership with the Brazilian Ministry of Planning, Budget, and Management | 43 | partnership with the Brazilian Ministry of Planning, Budget, and Management |
| 44 | -(MP) designed an integrated platform for collaborative software development | 44 | +designed an integrated platform for collaborative software development |
| 45 | \cite{bobr2003}, including social networking, mailing lists, version control | 45 | \cite{bobr2003}, including social networking, mailing lists, version control |
| 46 | system, and source code quality monitoring. To coordinate and develop this | 46 | system, and source code quality monitoring. To coordinate and develop this |
| 47 | project during 30 months, UnB was funded by a grant of 2,619,965.00 BRL (about | 47 | project during 30 months, UnB was funded by a grant of 2,619,965.00 BRL (about |
opensym2017/content/04-researchdesign.tex
| @@ -6,7 +6,7 @@ Portal by reporting the technical efforts carried out, our empirical work | @@ -6,7 +6,7 @@ Portal by reporting the technical efforts carried out, our empirical work | ||
| 6 | process, and the lessons learned. The new SPB Portal project presented three | 6 | process, and the lessons learned. The new SPB Portal project presented three |
| 7 | main challenges, related to the open questions described below. | 7 | main challenges, related to the open questions described below. |
| 8 | 8 | ||
| 9 | -\textbf{Q1:} \textit{Which strategy could be used to integrate several existing | 9 | +\textbf{Q1.} \textit{Which strategy could be used to integrate several existing |
| 10 | FLOSS tools to promote a collaborative software development?} | 10 | FLOSS tools to promote a collaborative software development?} |
| 11 | % | 11 | % |
| 12 | Based on an extensive list of functional requirements defined by the Brazilian | 12 | Based on an extensive list of functional requirements defined by the Brazilian |
| @@ -17,7 +17,7 @@ were fully aware that we would need to improve those systems in order to | @@ -17,7 +17,7 @@ were fully aware that we would need to improve those systems in order to | ||
| 17 | satisfy the remaining requirements. We were also convinced that it would be | 17 | satisfy the remaining requirements. We were also convinced that it would be |
| 18 | impossible to satisfy all of those requirements with a single tool. | 18 | impossible to satisfy all of those requirements with a single tool. |
| 19 | 19 | ||
| 20 | -\textbf{Q2:} \textit{How to involve students in real-world projects interacting | 20 | +\textbf{Q2.} \textit{How to involve students in real-world projects interacting |
| 21 | with real customers?} | 21 | with real customers?} |
| 22 | % | 22 | % |
| 23 | Our team was mainly composed of software engineering undergraduate students, | 23 | Our team was mainly composed of software engineering undergraduate students, |
| @@ -27,7 +27,7 @@ Government. For the majority of the students, this was a first professional | @@ -27,7 +27,7 @@ Government. For the majority of the students, this was a first professional | ||
| 27 | experience. Even though, our development process defined a central role on | 27 | experience. Even though, our development process defined a central role on |
| 28 | students participation. | 28 | students participation. |
| 29 | 29 | ||
| 30 | -\textbf{Q3:} \textit{How to introduce typical FLOSS collaborative and agile practices in the governmental development process?} | 30 | +\textbf{Q3.} \textit{How to introduce typical FLOSS collaborative and agile practices in the governmental development process?} |
| 31 | % | 31 | % |
| 32 | The software development in Brazilian government is based on a very traditional | 32 | The software development in Brazilian government is based on a very traditional |
| 33 | way, frequently focusing documentation deliveries. We had to convince them to | 33 | way, frequently focusing documentation deliveries. We had to convince them to |
opensym2017/content/06-architecture.tex
| @@ -159,7 +159,7 @@ network created between them. | @@ -159,7 +159,7 @@ network created between them. | ||
| 159 | 159 | ||
| 160 | \begin{figure*}[hbt] | 160 | \begin{figure*}[hbt] |
| 161 | \centering | 161 | \centering |
| 162 | - \includegraphics[width=\linewidth]{figures/arch3.png} | 162 | + \includegraphics[width=.85\linewidth]{figures/arch3.png} |
| 163 | \caption{Instanciation view of the SPB architecture.} | 163 | \caption{Instanciation view of the SPB architecture.} |
| 164 | \label{fig:architecture2} | 164 | \label{fig:architecture2} |
| 165 | \end{figure*} | 165 | \end{figure*} |
opensym2017/content/08-process.tex
| @@ -24,8 +24,8 @@ automation resulting from DevOps practices. Thus, the work process was executed | @@ -24,8 +24,8 @@ automation resulting from DevOps practices. Thus, the work process was executed | ||
| 24 | in a cadenced and continuous way. | 24 | in a cadenced and continuous way. |
| 25 | 25 | ||
| 26 | Finally, the last group of actors of this project was composed of employees of | 26 | Finally, the last group of actors of this project was composed of employees of |
| 27 | -the Brazilian Ministry of Planning, Development, and Management (MP is the | ||
| 28 | -Brazilian acronym). All the project decisions, validations, and scope | 27 | +the Brazilian Ministry of Planning, Development, and Management. |
| 28 | +All the project decisions, validations, and scope | ||
| 29 | definitions were made by them. In this way, we incrementally developed a | 29 | definitions were made by them. In this way, we incrementally developed a |
| 30 | software product with releases aligned to strategic business objectives. As one | 30 | software product with releases aligned to strategic business objectives. As one |
| 31 | can see, the project had a wide range of different stakeholders that had to be | 31 | can see, the project had a wide range of different stakeholders that had to be |
opensym2017/content/09-conclusion.tex
| @@ -4,20 +4,20 @@ | @@ -4,20 +4,20 @@ | ||
| 4 | In this work, we presented and discussed issues experienced during a | 4 | In this work, we presented and discussed issues experienced during a |
| 5 | government-funded project, in partnership with the University of Brasilia and | 5 | government-funded project, in partnership with the University of Brasilia and |
| 6 | the University of São Paulo, to evolve the Brazilian Public Software Portal. | 6 | the University of São Paulo, to evolve the Brazilian Public Software Portal. |
| 7 | -Its contributions are twofold. First, we present the strategy used to develop | 7 | +Our contributions are twofold. First, we present the strategy used to develop |
| 8 | and to deliver an unprecedented platform to Brazilian government. Second, based | 8 | and to deliver an unprecedented platform to Brazilian government. Second, based |
| 9 | on the results of the SPB Portal project, we point out that it is possible to | 9 | on the results of the SPB Portal project, we point out that it is possible to |
| 10 | mitigate conflicts experienced in the development environment and to conciliate | 10 | mitigate conflicts experienced in the development environment and to conciliate |
| 11 | governmental and academy cultures. To summarize our main contributions, we | 11 | governmental and academy cultures. To summarize our main contributions, we |
| 12 | answered in this section the three open questions those guided this paper. | 12 | answered in this section the three open questions those guided this paper. |
| 13 | 13 | ||
| 14 | -\textbf{Which strategy could be used to integrate several existing | ||
| 15 | -FLOSS tools to promote a collaborative software development?} | 14 | +\textbf{Which strategy could be used to integrate several existing FLOSS tools |
| 15 | +to promote a collaborative software development?} | ||
| 16 | % | 16 | % |
| 17 | -The SPB portal integrates more than 10 FLOSS tools and provides several | 17 | +The SPB portal integrates more than ten FLOSS tools and provides several |
| 18 | features, such as social network, mailing list, version control, content | 18 | features, such as social network, mailing list, version control, content |
| 19 | management and source code quality monitoring. Concerned with the platform | 19 | management and source code quality monitoring. Concerned with the platform |
| 20 | -sustainability and maintainability, the aforementioned 10 FLOSS tools were | 20 | +sustainability and maintainability, the aforementioned FLOSS tools were |
| 21 | integrated with minimum differences from their official versions and the new | 21 | integrated with minimum differences from their official versions and the new |
| 22 | developed features were sent upstream to ensure an alignment between the portal | 22 | developed features were sent upstream to ensure an alignment between the portal |
| 23 | systems and their respective official versions. In the integration process, the | 23 | systems and their respective official versions. In the integration process, the |
| @@ -25,8 +25,8 @@ main softwares were identified, specific teams were formed to work with each | @@ -25,8 +25,8 @@ main softwares were identified, specific teams were formed to work with each | ||
| 25 | one of them and each team was composed of students with different levels of | 25 | one of them and each team was composed of students with different levels of |
| 26 | skills and at least one senior professional. | 26 | skills and at least one senior professional. |
| 27 | 27 | ||
| 28 | -\textbf{How to involve students in real-world projects interacting with | ||
| 29 | -real customers?} | 28 | +\textbf{How to involve students in real-world projects interacting with real |
| 29 | +customers?} | ||
| 30 | % | 30 | % |
| 31 | In terms of mitigating conflicts, we tried to show that, as long as the | 31 | In terms of mitigating conflicts, we tried to show that, as long as the |
| 32 | university can provide a healthy and challenging environment to its students, | 32 | university can provide a healthy and challenging environment to its students, |
| @@ -42,29 +42,31 @@ professionals to solve development issues. | @@ -42,29 +42,31 @@ professionals to solve development issues. | ||
| 42 | % | 42 | % |
| 43 | Among the students, we have defined coaches for each team and a meta-coach | 43 | Among the students, we have defined coaches for each team and a meta-coach |
| 44 | (coach of whole project). All coaches, together with professors, have | 44 | (coach of whole project). All coaches, together with professors, have |
| 45 | -intermediated the communication between client (Ministry of Planning of Brazil) | 45 | +intermediated the communication between client (the Brazilian Government) |
| 46 | and the rest of the group. | 46 | and the rest of the group. |
| 47 | After the end of the project, some students successfully | 47 | After the end of the project, some students successfully |
| 48 | embraced opportunities in public and private sectors, within national borders | 48 | embraced opportunities in public and private sectors, within national borders |
| 49 | and abroad. Some other students went further and started their own companies. | 49 | and abroad. Some other students went further and started their own companies. |
| 50 | 50 | ||
| 51 | -\textbf{How to introduce collaborative and agile | ||
| 52 | -practices typical in FLOSS environments in the governmental development process?} | 51 | +\textbf{How to introduce typical FLOSS collaborative and agile practices in the |
| 52 | +governmental development process?} | ||
| 53 | % | 53 | % |
| 54 | -With some adaptations, what we called the ``translation processes'', it is | 54 | +With some adaptations, it is |
| 55 | feasible to conciliate agile methodologies and FLOSS practices to develop | 55 | feasible to conciliate agile methodologies and FLOSS practices to develop |
| 56 | software to governmental organizations with functional hierarchical structures | 56 | software to governmental organizations with functional hierarchical structures |
| 57 | that use traditional development paradigm. | 57 | that use traditional development paradigm. |
| 58 | % | 58 | % |
| 59 | -Aiming at reducing client questions about workconclusion, a DevOps front was | 59 | +Aiming at reducing client questions about workconclusion, a DevOps team was |
| 60 | created to automate all deploy process and also to work in continuous | 60 | created to automate all deploy process and also to work in continuous |
| 61 | -delivery. The government was brought to our work environment and interacted | 61 | +delivery. The Government was brought to our work environment and interacted |
| 62 | with our management and communication tools. For the project success, we | 62 | with our management and communication tools. For the project success, we |
| 63 | focused on providing a friendly working environment as well as on showing to | 63 | focused on providing a friendly working environment as well as on showing to |
| 64 | governmental agents another way to interact with the FLOSS community and the | 64 | governmental agents another way to interact with the FLOSS community and the |
| 65 | university. | 65 | university. |
| 66 | 66 | ||
| 67 | -From the answers of our initial open questions, we can also highlighted six | 67 | +\subsection{Lessons Learned} |
| 68 | + | ||
| 69 | +From the answers of our initial open questions, we also can highlighted six | ||
| 68 | lessons learned to make easier to share our experience during the development | 70 | lessons learned to make easier to share our experience during the development |
| 69 | of the new SPB Portal. | 71 | of the new SPB Portal. |
| 70 | 72 | ||
| @@ -104,21 +106,22 @@ certain expectations about the project development that, let's say, did not | @@ -104,21 +106,22 @@ certain expectations about the project development that, let's say, did not | ||
| 104 | exactly match our work style based on agile and FLOSS practices. | 106 | exactly match our work style based on agile and FLOSS practices. |
| 105 | % | 107 | % |
| 106 | We had to develop strategies that would support these different organizational | 108 | We had to develop strategies that would support these different organizational |
| 107 | -cultures. Therefore, we have created ``translation processes'' between our team | ||
| 108 | -and the MP managers who managed the project on their side, assuming a | 109 | +cultures. Therefore, we have adapted the process between our team |
| 110 | +and the Government managers who managed the project on their side, assuming a | ||
| 109 | traditional waterfall process. | 111 | traditional waterfall process. |
| 110 | 112 | ||
| 111 | \textbf{Managing higher-level and lower-level goals separately.} | 113 | \textbf{Managing higher-level and lower-level goals separately.} |
| 112 | % | 114 | % |
| 113 | -During the initial phase of the project, the MP team has brought strategic | ||
| 114 | -discussions to technical/operational meetings that were supposed to be about | ||
| 115 | -practical technical decisions. | 115 | +During the initial phase of the project, the Brazilian Government team has |
| 116 | +brought strategic discussions to technical/operational meetings that were | ||
| 117 | +supposed to be about practical technical decisions. | ||
| 116 | % | 118 | % |
| 117 | This produced a highly complex communication and management environment, | 119 | This produced a highly complex communication and management environment, |
| 118 | -overloading the professors because they were supposed for maintaining the MP | ||
| 119 | -strategy synchronized with the implementation plans of the development team. | ||
| 120 | -This was hard, especially because the aforementioned cultural mismatch. Mixing | ||
| 121 | -both concerns in the same discussions caused confusion on both sides. | 120 | +overloading the professors because they were supposed for maintaining the |
| 121 | +Government strategy synchronized with the implementation plans of the | ||
| 122 | +development team. This was hard, especially because the aforementioned | ||
| 123 | +cultural mismatch. Mixing both concerns in the same discussions caused | ||
| 124 | +confusion on both sides. | ||
| 122 | % | 125 | % |
| 123 | From the middle of the project we were able to keep those concerns separated, | 126 | From the middle of the project we were able to keep those concerns separated, |
| 124 | what eased the work of everyone involved. | 127 | what eased the work of everyone involved. |
| @@ -126,40 +129,41 @@ what eased the work of everyone involved. | @@ -126,40 +129,41 @@ what eased the work of everyone involved. | ||
| 126 | \textbf{Living with ill-advised political decisions.} | 129 | \textbf{Living with ill-advised political decisions.} |
| 127 | % | 130 | % |
| 128 | At the initial phases of the project, by political and personal motivation, the | 131 | At the initial phases of the project, by political and personal motivation, the |
| 129 | -main stakeholders from the Brazilian government imposed the use of Colab to | 132 | +main stakeholders from the Brazilian Government imposed the use of Colab to |
| 130 | guide the development of the new SPB platform. Our team was totally against the | 133 | guide the development of the new SPB platform. Our team was totally against the |
| 131 | idea because we already knew that Colab was a very experimental project and its | 134 | idea because we already knew that Colab was a very experimental project and its |
| 132 | -adoption could dramatically increase the project complexity. Even though we | ||
| 133 | -provided technical reasons to not utilize Colab, the MP was adamant and we had | ||
| 134 | -to manage this problem. We did massive changes to Colab, and at the end of the | ||
| 135 | -project we have completely rewritten it to make it stable. It is important to | ||
| 136 | -notice that the MP compelled us to accept a technical decision based only on | ||
| 137 | -political interests, without considering all the resources that would be spent | ||
| 138 | -due to that decision. At the end of the project, we verified that Colab indeed | ||
| 139 | -consumed a vast amount of the budget and increased the project complexity. At | ||
| 140 | -the end of the project and after our analysis on the decision made by the MP, | ||
| 141 | -we understand that MP managers are capable of ignoring technical reasons in | ||
| 142 | -favor of political decisions. | 135 | +adoption could dramatically increase the project complexity. Even though, we |
| 136 | +provided technical reasons to not utilize Colab, the Government was adamant and | ||
| 137 | +we had to manage this problem. We did massive changes to Colab, and at the end | ||
| 138 | +of the project we have completely rewritten it to make it stable. It is | ||
| 139 | +important to notice that the Government compelled us to accept a technical | ||
| 140 | +decision based only on political interests, without considering all the | ||
| 141 | +resources that would be spent due to that decision. At the end of the project, | ||
| 142 | +we verified that Colab indeed consumed a vast amount of the budget and | ||
| 143 | +increased the project complexity. After our analysis on the decision made by | ||
| 144 | +the Government, we understand that some Brazilian Government managers are | ||
| 145 | +capable of ignoring technical reasons in favor of political decisions. | ||
| 143 | 146 | ||
| 144 | \textbf{Consider sustainability from the beginning.} | 147 | \textbf{Consider sustainability from the beginning.} |
| 145 | % | 148 | % |
| 146 | -In the process of deploying the SPB platform in the MP infrastructure we had to | ||
| 147 | -interact with the MP technicians. We did several workshops, training and a | ||
| 148 | -meticulous documentation describing all the required procedures to update the | ||
| 149 | -platform, however, we realized that the MP technicians constantly avoid the | ||
| 150 | -responsibility. After noticing the aforementioned situation, we organized a | ||
| 151 | -DevOps team that completely automated all the deployment procedure. We | ||
| 152 | -simplified all the platform deployment to a few steps: (1) initial | ||
| 153 | -configurations (just ssh configuration) and (2) the execution of simple | ||
| 154 | -commands to completely update the platform. By the end of the project, we | ||
| 155 | -observed that the MP technicians invariably still depended on our support to | ||
| 156 | -update the platform even with all the automation provided by us. We were sadly | ||
| 157 | -left with a feeling of uncertainty about the future of the platform after the | ||
| 158 | -project ended. In hindsight, we realize that the MP dedicated system analysts | ||
| 159 | -and managers to the project, but not operations technicians. The later should | ||
| 160 | -have been more involved with the process so they could at least be comfortable | ||
| 161 | -in managing the platform infrastructure. | ||
| 162 | - | 149 | +In the process of deploying the SPB platform in the Brazilian Government |
| 150 | +infrastructure we had to interact with the Government technicians. We did | ||
| 151 | +several workshops, training and a meticulous documentation describing all the | ||
| 152 | +required procedures to update the platform, however, we realized that they | ||
| 153 | +constantly avoid the responsibility. After noticing the aforementioned | ||
| 154 | +situation, we organized a DevOps team that completely automated all the | ||
| 155 | +deployment procedure. We simplified all the platform deployment to a few steps: | ||
| 156 | +(i) initial configurations (just ssh configuration) and (ii) the execution of | ||
| 157 | +simple commands to completely update the platform. By the end of the project, | ||
| 158 | +we observed that the Government technicians invariably still depended on our | ||
| 159 | +support to update the platform even with all the automation provided by us. We | ||
| 160 | +were sadly left with a feeling of uncertainty about the future of the platform | ||
| 161 | +after the project ended. In hindsight, we realize that the Brazilian Government | ||
| 162 | +dedicated system analysts and managers to the project, but not operations | ||
| 163 | +technicians. The later should have been more involved with the process so they | ||
| 164 | +could at least be comfortable in managing the platform infrastructure. | ||
| 165 | + | ||
| 166 | +\subsection{Final Remarks and Future Work} | ||
| 163 | 167 | ||
| 164 | Ultimately, the SPB portal is in | 168 | Ultimately, the SPB portal is in |
| 165 | production\footnote{\url{https://softwarepublico.gov.br}} and its full | 169 | production\footnote{\url{https://softwarepublico.gov.br}} and its full |