Commit db4810a248c190845821d098c7e891a7418dd038
1 parent
6dd0b976
Exists in
master
and in
3 other branches
[opensym] Reviewing the conclusion
Showing
5 changed files
with
63 additions
and
59 deletions
Show diff stats
opensym2017/content/01-introduction.tex
@@ -41,7 +41,7 @@ Approximately five years later from the first problems, a new platform for the | @@ -41,7 +41,7 @@ Approximately five years later from the first problems, a new platform for the | ||
41 | SPB Portal was planned and funded. Between January 2014 and June 2016, the | 41 | SPB Portal was planned and funded. Between January 2014 and June 2016, the |
42 | University of Brasília (UnB) and the University of São Paulo (USP) in a | 42 | University of Brasília (UnB) and the University of São Paulo (USP) in a |
43 | partnership with the Brazilian Ministry of Planning, Budget, and Management | 43 | partnership with the Brazilian Ministry of Planning, Budget, and Management |
44 | -(MP) designed an integrated platform for collaborative software development | 44 | +designed an integrated platform for collaborative software development |
45 | \cite{bobr2003}, including social networking, mailing lists, version control | 45 | \cite{bobr2003}, including social networking, mailing lists, version control |
46 | system, and source code quality monitoring. To coordinate and develop this | 46 | system, and source code quality monitoring. To coordinate and develop this |
47 | project during 30 months, UnB was funded by a grant of 2,619,965.00 BRL (about | 47 | project during 30 months, UnB was funded by a grant of 2,619,965.00 BRL (about |
opensym2017/content/04-researchdesign.tex
@@ -6,7 +6,7 @@ Portal by reporting the technical efforts carried out, our empirical work | @@ -6,7 +6,7 @@ Portal by reporting the technical efforts carried out, our empirical work | ||
6 | process, and the lessons learned. The new SPB Portal project presented three | 6 | process, and the lessons learned. The new SPB Portal project presented three |
7 | main challenges, related to the open questions described below. | 7 | main challenges, related to the open questions described below. |
8 | 8 | ||
9 | -\textbf{Q1:} \textit{Which strategy could be used to integrate several existing | 9 | +\textbf{Q1.} \textit{Which strategy could be used to integrate several existing |
10 | FLOSS tools to promote a collaborative software development?} | 10 | FLOSS tools to promote a collaborative software development?} |
11 | % | 11 | % |
12 | Based on an extensive list of functional requirements defined by the Brazilian | 12 | Based on an extensive list of functional requirements defined by the Brazilian |
@@ -17,7 +17,7 @@ were fully aware that we would need to improve those systems in order to | @@ -17,7 +17,7 @@ were fully aware that we would need to improve those systems in order to | ||
17 | satisfy the remaining requirements. We were also convinced that it would be | 17 | satisfy the remaining requirements. We were also convinced that it would be |
18 | impossible to satisfy all of those requirements with a single tool. | 18 | impossible to satisfy all of those requirements with a single tool. |
19 | 19 | ||
20 | -\textbf{Q2:} \textit{How to involve students in real-world projects interacting | 20 | +\textbf{Q2.} \textit{How to involve students in real-world projects interacting |
21 | with real customers?} | 21 | with real customers?} |
22 | % | 22 | % |
23 | Our team was mainly composed of software engineering undergraduate students, | 23 | Our team was mainly composed of software engineering undergraduate students, |
@@ -27,7 +27,7 @@ Government. For the majority of the students, this was a first professional | @@ -27,7 +27,7 @@ Government. For the majority of the students, this was a first professional | ||
27 | experience. Even though, our development process defined a central role on | 27 | experience. Even though, our development process defined a central role on |
28 | students participation. | 28 | students participation. |
29 | 29 | ||
30 | -\textbf{Q3:} \textit{How to introduce typical FLOSS collaborative and agile practices in the governmental development process?} | 30 | +\textbf{Q3.} \textit{How to introduce typical FLOSS collaborative and agile practices in the governmental development process?} |
31 | % | 31 | % |
32 | The software development in Brazilian government is based on a very traditional | 32 | The software development in Brazilian government is based on a very traditional |
33 | way, frequently focusing documentation deliveries. We had to convince them to | 33 | way, frequently focusing documentation deliveries. We had to convince them to |
opensym2017/content/06-architecture.tex
@@ -159,7 +159,7 @@ network created between them. | @@ -159,7 +159,7 @@ network created between them. | ||
159 | 159 | ||
160 | \begin{figure*}[hbt] | 160 | \begin{figure*}[hbt] |
161 | \centering | 161 | \centering |
162 | - \includegraphics[width=\linewidth]{figures/arch3.png} | 162 | + \includegraphics[width=.85\linewidth]{figures/arch3.png} |
163 | \caption{Instanciation view of the SPB architecture.} | 163 | \caption{Instanciation view of the SPB architecture.} |
164 | \label{fig:architecture2} | 164 | \label{fig:architecture2} |
165 | \end{figure*} | 165 | \end{figure*} |
opensym2017/content/08-process.tex
@@ -24,8 +24,8 @@ automation resulting from DevOps practices. Thus, the work process was executed | @@ -24,8 +24,8 @@ automation resulting from DevOps practices. Thus, the work process was executed | ||
24 | in a cadenced and continuous way. | 24 | in a cadenced and continuous way. |
25 | 25 | ||
26 | Finally, the last group of actors of this project was composed of employees of | 26 | Finally, the last group of actors of this project was composed of employees of |
27 | -the Brazilian Ministry of Planning, Development, and Management (MP is the | ||
28 | -Brazilian acronym). All the project decisions, validations, and scope | 27 | +the Brazilian Ministry of Planning, Development, and Management. |
28 | +All the project decisions, validations, and scope | ||
29 | definitions were made by them. In this way, we incrementally developed a | 29 | definitions were made by them. In this way, we incrementally developed a |
30 | software product with releases aligned to strategic business objectives. As one | 30 | software product with releases aligned to strategic business objectives. As one |
31 | can see, the project had a wide range of different stakeholders that had to be | 31 | can see, the project had a wide range of different stakeholders that had to be |
opensym2017/content/09-conclusion.tex
@@ -4,20 +4,20 @@ | @@ -4,20 +4,20 @@ | ||
4 | In this work, we presented and discussed issues experienced during a | 4 | In this work, we presented and discussed issues experienced during a |
5 | government-funded project, in partnership with the University of Brasilia and | 5 | government-funded project, in partnership with the University of Brasilia and |
6 | the University of São Paulo, to evolve the Brazilian Public Software Portal. | 6 | the University of São Paulo, to evolve the Brazilian Public Software Portal. |
7 | -Its contributions are twofold. First, we present the strategy used to develop | 7 | +Our contributions are twofold. First, we present the strategy used to develop |
8 | and to deliver an unprecedented platform to Brazilian government. Second, based | 8 | and to deliver an unprecedented platform to Brazilian government. Second, based |
9 | on the results of the SPB Portal project, we point out that it is possible to | 9 | on the results of the SPB Portal project, we point out that it is possible to |
10 | mitigate conflicts experienced in the development environment and to conciliate | 10 | mitigate conflicts experienced in the development environment and to conciliate |
11 | governmental and academy cultures. To summarize our main contributions, we | 11 | governmental and academy cultures. To summarize our main contributions, we |
12 | answered in this section the three open questions those guided this paper. | 12 | answered in this section the three open questions those guided this paper. |
13 | 13 | ||
14 | -\textbf{Which strategy could be used to integrate several existing | ||
15 | -FLOSS tools to promote a collaborative software development?} | 14 | +\textbf{Which strategy could be used to integrate several existing FLOSS tools |
15 | +to promote a collaborative software development?} | ||
16 | % | 16 | % |
17 | -The SPB portal integrates more than 10 FLOSS tools and provides several | 17 | +The SPB portal integrates more than ten FLOSS tools and provides several |
18 | features, such as social network, mailing list, version control, content | 18 | features, such as social network, mailing list, version control, content |
19 | management and source code quality monitoring. Concerned with the platform | 19 | management and source code quality monitoring. Concerned with the platform |
20 | -sustainability and maintainability, the aforementioned 10 FLOSS tools were | 20 | +sustainability and maintainability, the aforementioned FLOSS tools were |
21 | integrated with minimum differences from their official versions and the new | 21 | integrated with minimum differences from their official versions and the new |
22 | developed features were sent upstream to ensure an alignment between the portal | 22 | developed features were sent upstream to ensure an alignment between the portal |
23 | systems and their respective official versions. In the integration process, the | 23 | systems and their respective official versions. In the integration process, the |
@@ -25,8 +25,8 @@ main softwares were identified, specific teams were formed to work with each | @@ -25,8 +25,8 @@ main softwares were identified, specific teams were formed to work with each | ||
25 | one of them and each team was composed of students with different levels of | 25 | one of them and each team was composed of students with different levels of |
26 | skills and at least one senior professional. | 26 | skills and at least one senior professional. |
27 | 27 | ||
28 | -\textbf{How to involve students in real-world projects interacting with | ||
29 | -real customers?} | 28 | +\textbf{How to involve students in real-world projects interacting with real |
29 | +customers?} | ||
30 | % | 30 | % |
31 | In terms of mitigating conflicts, we tried to show that, as long as the | 31 | In terms of mitigating conflicts, we tried to show that, as long as the |
32 | university can provide a healthy and challenging environment to its students, | 32 | university can provide a healthy and challenging environment to its students, |
@@ -42,29 +42,31 @@ professionals to solve development issues. | @@ -42,29 +42,31 @@ professionals to solve development issues. | ||
42 | % | 42 | % |
43 | Among the students, we have defined coaches for each team and a meta-coach | 43 | Among the students, we have defined coaches for each team and a meta-coach |
44 | (coach of whole project). All coaches, together with professors, have | 44 | (coach of whole project). All coaches, together with professors, have |
45 | -intermediated the communication between client (Ministry of Planning of Brazil) | 45 | +intermediated the communication between client (the Brazilian Government) |
46 | and the rest of the group. | 46 | and the rest of the group. |
47 | After the end of the project, some students successfully | 47 | After the end of the project, some students successfully |
48 | embraced opportunities in public and private sectors, within national borders | 48 | embraced opportunities in public and private sectors, within national borders |
49 | and abroad. Some other students went further and started their own companies. | 49 | and abroad. Some other students went further and started their own companies. |
50 | 50 | ||
51 | -\textbf{How to introduce collaborative and agile | ||
52 | -practices typical in FLOSS environments in the governmental development process?} | 51 | +\textbf{How to introduce typical FLOSS collaborative and agile practices in the |
52 | +governmental development process?} | ||
53 | % | 53 | % |
54 | -With some adaptations, what we called the ``translation processes'', it is | 54 | +With some adaptations, it is |
55 | feasible to conciliate agile methodologies and FLOSS practices to develop | 55 | feasible to conciliate agile methodologies and FLOSS practices to develop |
56 | software to governmental organizations with functional hierarchical structures | 56 | software to governmental organizations with functional hierarchical structures |
57 | that use traditional development paradigm. | 57 | that use traditional development paradigm. |
58 | % | 58 | % |
59 | -Aiming at reducing client questions about workconclusion, a DevOps front was | 59 | +Aiming at reducing client questions about workconclusion, a DevOps team was |
60 | created to automate all deploy process and also to work in continuous | 60 | created to automate all deploy process and also to work in continuous |
61 | -delivery. The government was brought to our work environment and interacted | 61 | +delivery. The Government was brought to our work environment and interacted |
62 | with our management and communication tools. For the project success, we | 62 | with our management and communication tools. For the project success, we |
63 | focused on providing a friendly working environment as well as on showing to | 63 | focused on providing a friendly working environment as well as on showing to |
64 | governmental agents another way to interact with the FLOSS community and the | 64 | governmental agents another way to interact with the FLOSS community and the |
65 | university. | 65 | university. |
66 | 66 | ||
67 | -From the answers of our initial open questions, we can also highlighted six | 67 | +\subsection{Lessons Learned} |
68 | + | ||
69 | +From the answers of our initial open questions, we also can highlighted six | ||
68 | lessons learned to make easier to share our experience during the development | 70 | lessons learned to make easier to share our experience during the development |
69 | of the new SPB Portal. | 71 | of the new SPB Portal. |
70 | 72 | ||
@@ -104,21 +106,22 @@ certain expectations about the project development that, let's say, did not | @@ -104,21 +106,22 @@ certain expectations about the project development that, let's say, did not | ||
104 | exactly match our work style based on agile and FLOSS practices. | 106 | exactly match our work style based on agile and FLOSS practices. |
105 | % | 107 | % |
106 | We had to develop strategies that would support these different organizational | 108 | We had to develop strategies that would support these different organizational |
107 | -cultures. Therefore, we have created ``translation processes'' between our team | ||
108 | -and the MP managers who managed the project on their side, assuming a | 109 | +cultures. Therefore, we have adapted the process between our team |
110 | +and the Government managers who managed the project on their side, assuming a | ||
109 | traditional waterfall process. | 111 | traditional waterfall process. |
110 | 112 | ||
111 | \textbf{Managing higher-level and lower-level goals separately.} | 113 | \textbf{Managing higher-level and lower-level goals separately.} |
112 | % | 114 | % |
113 | -During the initial phase of the project, the MP team has brought strategic | ||
114 | -discussions to technical/operational meetings that were supposed to be about | ||
115 | -practical technical decisions. | 115 | +During the initial phase of the project, the Brazilian Government team has |
116 | +brought strategic discussions to technical/operational meetings that were | ||
117 | +supposed to be about practical technical decisions. | ||
116 | % | 118 | % |
117 | This produced a highly complex communication and management environment, | 119 | This produced a highly complex communication and management environment, |
118 | -overloading the professors because they were supposed for maintaining the MP | ||
119 | -strategy synchronized with the implementation plans of the development team. | ||
120 | -This was hard, especially because the aforementioned cultural mismatch. Mixing | ||
121 | -both concerns in the same discussions caused confusion on both sides. | 120 | +overloading the professors because they were supposed for maintaining the |
121 | +Government strategy synchronized with the implementation plans of the | ||
122 | +development team. This was hard, especially because the aforementioned | ||
123 | +cultural mismatch. Mixing both concerns in the same discussions caused | ||
124 | +confusion on both sides. | ||
122 | % | 125 | % |
123 | From the middle of the project we were able to keep those concerns separated, | 126 | From the middle of the project we were able to keep those concerns separated, |
124 | what eased the work of everyone involved. | 127 | what eased the work of everyone involved. |
@@ -126,40 +129,41 @@ what eased the work of everyone involved. | @@ -126,40 +129,41 @@ what eased the work of everyone involved. | ||
126 | \textbf{Living with ill-advised political decisions.} | 129 | \textbf{Living with ill-advised political decisions.} |
127 | % | 130 | % |
128 | At the initial phases of the project, by political and personal motivation, the | 131 | At the initial phases of the project, by political and personal motivation, the |
129 | -main stakeholders from the Brazilian government imposed the use of Colab to | 132 | +main stakeholders from the Brazilian Government imposed the use of Colab to |
130 | guide the development of the new SPB platform. Our team was totally against the | 133 | guide the development of the new SPB platform. Our team was totally against the |
131 | idea because we already knew that Colab was a very experimental project and its | 134 | idea because we already knew that Colab was a very experimental project and its |
132 | -adoption could dramatically increase the project complexity. Even though we | ||
133 | -provided technical reasons to not utilize Colab, the MP was adamant and we had | ||
134 | -to manage this problem. We did massive changes to Colab, and at the end of the | ||
135 | -project we have completely rewritten it to make it stable. It is important to | ||
136 | -notice that the MP compelled us to accept a technical decision based only on | ||
137 | -political interests, without considering all the resources that would be spent | ||
138 | -due to that decision. At the end of the project, we verified that Colab indeed | ||
139 | -consumed a vast amount of the budget and increased the project complexity. At | ||
140 | -the end of the project and after our analysis on the decision made by the MP, | ||
141 | -we understand that MP managers are capable of ignoring technical reasons in | ||
142 | -favor of political decisions. | 135 | +adoption could dramatically increase the project complexity. Even though, we |
136 | +provided technical reasons to not utilize Colab, the Government was adamant and | ||
137 | +we had to manage this problem. We did massive changes to Colab, and at the end | ||
138 | +of the project we have completely rewritten it to make it stable. It is | ||
139 | +important to notice that the Government compelled us to accept a technical | ||
140 | +decision based only on political interests, without considering all the | ||
141 | +resources that would be spent due to that decision. At the end of the project, | ||
142 | +we verified that Colab indeed consumed a vast amount of the budget and | ||
143 | +increased the project complexity. After our analysis on the decision made by | ||
144 | +the Government, we understand that some Brazilian Government managers are | ||
145 | +capable of ignoring technical reasons in favor of political decisions. | ||
143 | 146 | ||
144 | \textbf{Consider sustainability from the beginning.} | 147 | \textbf{Consider sustainability from the beginning.} |
145 | % | 148 | % |
146 | -In the process of deploying the SPB platform in the MP infrastructure we had to | ||
147 | -interact with the MP technicians. We did several workshops, training and a | ||
148 | -meticulous documentation describing all the required procedures to update the | ||
149 | -platform, however, we realized that the MP technicians constantly avoid the | ||
150 | -responsibility. After noticing the aforementioned situation, we organized a | ||
151 | -DevOps team that completely automated all the deployment procedure. We | ||
152 | -simplified all the platform deployment to a few steps: (1) initial | ||
153 | -configurations (just ssh configuration) and (2) the execution of simple | ||
154 | -commands to completely update the platform. By the end of the project, we | ||
155 | -observed that the MP technicians invariably still depended on our support to | ||
156 | -update the platform even with all the automation provided by us. We were sadly | ||
157 | -left with a feeling of uncertainty about the future of the platform after the | ||
158 | -project ended. In hindsight, we realize that the MP dedicated system analysts | ||
159 | -and managers to the project, but not operations technicians. The later should | ||
160 | -have been more involved with the process so they could at least be comfortable | ||
161 | -in managing the platform infrastructure. | ||
162 | - | 149 | +In the process of deploying the SPB platform in the Brazilian Government |
150 | +infrastructure we had to interact with the Government technicians. We did | ||
151 | +several workshops, training and a meticulous documentation describing all the | ||
152 | +required procedures to update the platform, however, we realized that they | ||
153 | +constantly avoid the responsibility. After noticing the aforementioned | ||
154 | +situation, we organized a DevOps team that completely automated all the | ||
155 | +deployment procedure. We simplified all the platform deployment to a few steps: | ||
156 | +(i) initial configurations (just ssh configuration) and (ii) the execution of | ||
157 | +simple commands to completely update the platform. By the end of the project, | ||
158 | +we observed that the Government technicians invariably still depended on our | ||
159 | +support to update the platform even with all the automation provided by us. We | ||
160 | +were sadly left with a feeling of uncertainty about the future of the platform | ||
161 | +after the project ended. In hindsight, we realize that the Brazilian Government | ||
162 | +dedicated system analysts and managers to the project, but not operations | ||
163 | +technicians. The later should have been more involved with the process so they | ||
164 | +could at least be comfortable in managing the platform infrastructure. | ||
165 | + | ||
166 | +\subsection{Final Remarks and Future Work} | ||
163 | 167 | ||
164 | Ultimately, the SPB portal is in | 168 | Ultimately, the SPB portal is in |
165 | production\footnote{\url{https://softwarepublico.gov.br}} and its full | 169 | production\footnote{\url{https://softwarepublico.gov.br}} and its full |