Commit 892b9b0481e73d4bfa6ba7699d20b2d3b5b039ac
1 parent
fef7abc2
Exists in
master
and in
3 other branches
[oss-2018] Reviewing Results
Showing
1 changed file
with
108 additions
and
111 deletions
Show diff stats
icse2018/content/05-results.tex
1 | 1 | \section{Results} |
2 | 2 | \label{sec:results} |
3 | 3 | |
4 | -%TODO: Talvez esse paragráfo tem que está no Research Design | |
5 | -%% | |
6 | -The case study was analyzed and divided into two phases according to the project | |
7 | -management model. In the second phase (after one year of execution), several | |
8 | -practices have been applied to harmonize the cultural and organizational | |
9 | -divergences of the institutions involved. | |
10 | -%% | |
11 | -At the end of the project, an empirical | |
12 | -model of management and development process was created by aligning experiences | |
13 | -from the FLOSS universe, academic research and bureaucracies needed by the | |
14 | -government. In this section, we present by context the practices adopted in this | |
15 | -second phase and show the benefits generated by its deployment. | |
4 | +The case study had two phases according to the project management model. In the | |
5 | +second phase (about one year of execution), several practices have been applied | |
6 | +to harmonize the cultural and organizational divergences of the institutions | |
7 | +involved. At the end of the project, an empirical model of management and | |
8 | +development process was stabilized by aligning experiences from the open source | |
9 | +ecosystem, academic research, and bureaucracies needed by the government. In | |
10 | +this section, we present by context the practices adopted in this second phase | |
11 | +and show the benefits generated by its deployment. | |
16 | 12 | |
17 | 13 | \subsection{Project management and communication on the developing platform |
18 | 14 | itself} |
19 | 15 | |
20 | -After nine months of project activities, the first version of new SPB Portal was | |
21 | -released. From this point, we started to migrate the management and | |
22 | -communication interactions to the platform under development. In short, Wiki | |
23 | -feature was used for meeting logging, defining goals, sprint planning, and | |
24 | -documentation of deployment processes and administration resources guide. Issue | |
25 | -tracker was used for discussing requirements, monitoring the features under | |
26 | -development, registering changes, and validating functionalities delivered. Finally, the | |
27 | -whole team used Mailing list to defining schedules of meetings and deliveries | |
28 | -and also to collaborative definition of requirements. | |
16 | +After nine months of project activities, the first version of new SPB Portal | |
17 | +was released. From this point, we started to migrate the management and | |
18 | +communication interactions to the platform under development. In short, Wiki | |
19 | +feature was used for meeting logging, defining goals, sprint planning, and | |
20 | +documentation of deployment processes and administration resources guide. Issue | |
21 | +tracker was used for discussing requirements, monitoring the features under | |
22 | +development, registering changes, and validating functionalities delivered. | |
23 | +Finally, the whole team used Mailing list to defining schedules of meetings and | |
24 | +deliveries and also to collaborative definition of requirements. | |
29 | 25 | |
30 | -Our surveys reports Mailing list (100\%) and Issue Tracker (62.5\%) as the main means | |
31 | -of interaction between senior developers and undergraduates. Developers and MPOG | |
32 | -staff also interacted mostly via Mailing List (87.5\%) and Issue tracker (50\%). | |
33 | -According to research findings, this movement made \textbf{communication more | |
34 | -transparent and efficient}. A MPOG IT analyst said that the | |
35 | -\textit{``Communicating well goes far beyond the speed, it is someone being able | |
36 | -to communicate to everyone everything that is happening in the project. We did | |
37 | -not use emails. We use more mailing list and avoid e-mails. It helped a lot | |
38 | -because everything was public and did not pollute our mailbox. You wanted to | |
39 | -know something, could go there and look at what was happening''}. | |
26 | +Our surveys reports Mailing list (100\%) and Issue Tracker (62.5\%) as the main | |
27 | +means of interaction between senior developers and undergraduates. Developers | |
28 | +and MPOG staff also interacted mostly via Mailing List (87.5\%) and Issue | |
29 | +tracker (50\%). According to research findings, this movement made | |
30 | +\textbf{communication more transparent and efficient}. A MPOG IT analyst said | |
31 | +that the \textit{``Communicating well goes far beyond the speed, it is someone | |
32 | +being able to communicate to everyone everything that is happening in the | |
33 | +project. We did not use emails. We use more mailing list and avoid e-mails. It | |
34 | +helped a lot because everything was public and did not pollute our mailbox. You | |
35 | +wanted to know something, could go there and look at what was happening''}. | |
40 | 36 | |
41 | -Migrating to SPB platform also provided an \textbf{easier monitoring and | |
42 | -increase interactions between development team and public servants by | |
43 | -coordinators}. As shown by collected data, in the last 15 months of the project, | |
44 | -the issues have 59 different authors (8 from MPOG | |
45 | -staff), and commented by 64 different users (9 from MPOG staff and users). | |
46 | -Considering issues with higher level of interaction those that have 10 or more | |
47 | -comments, in a set of 102 issues, MPOG staff authored 43 issues (which represents | |
48 | -42\% of these most active issues). A MPOG analyst highlighted that | |
49 | -\textit{``there was a lot of evolution, a lot of communication via Gitlab''}. | |
50 | -This interaction also led MPOG staff to \textbf{trust developed code}: | |
51 | -\textit{``Everything was validated, we tested the features and the project was | |
52 | -developed inside the platform, so that the feature was validated in the | |
53 | -development of the software itself. From the moment we installed it, and | |
54 | -began to use it for development, this validation was constant. We felt confident | |
55 | -in the features''}. | |
37 | +Migrating to SPB platform also provided an \textbf{easier monitoring and | |
38 | +increase interactions between development team and public servants by | |
39 | +coordinators}. As shown by collected data, in the last 15 months of the | |
40 | +project, the issues have 59 different authors (8 from MPOG staff), and | |
41 | +commented by 64 different users (9 from MPOG staff and users). Considering | |
42 | +issues with higher level of interaction those that have 10 or more comments, in | |
43 | +a set of 102 issues, MPOG staff authored 43 issues (which represents 42\% of | |
44 | +these most active issues). A MPOG analyst highlighted that \textit{``there was | |
45 | +a lot of evolution, a lot of communication via Gitlab''}. This interaction | |
46 | +also led MPOG staff to \textbf{trust developed code}: \textit{``Everything was | |
47 | +validated, we tested the features and the project was developed inside the | |
48 | +platform, so that the feature was validated in the development of the software | |
49 | +itself. From the moment we installed it, and began to use it for development, | |
50 | +this validation was constant. We felt confident in the features''}. | |
56 | 51 | |
57 | -One of the main concerns of traditional approach is meticulous documentation of | |
52 | +One of the main concerns of traditional approach is meticulous documentation of | |
58 | 53 | the software designed and the development steps. With this aforementioned |
59 | -decision, we could meet this government demand without bureaucracies and changes | |
60 | -in our development process, \textbf{producting organically documentation and | |
61 | -records} in the platform itself, as one of the MPOG response evidenced: | |
62 | -\textit{``For me, it was a lot of learning. There is a lot of things documented | |
63 | -in the e-mails and also in the portal itself. At any moment we can go there and | |
64 | -see how it worked, how someone did something. We can recover those good points''}. | |
54 | +decision, we could meet this government demand without bureaucracies and | |
55 | +changes in our development process, \textbf{producting organically | |
56 | +documentation and records} in the platform itself, as one of the MPOG response | |
57 | +evidenced: \textit{``For me, it was a lot of learning. There is a lot of things | |
58 | +documented in the e-mails and also in the portal itself. At any moment we can | |
59 | +go there and see how it worked, how someone did something. We can recover those | |
60 | +good points''}. | |
65 | 61 | |
66 | 62 | \subsection{Bringing together government staff and development team} |
67 | 63 | |
... | ... | @@ -69,18 +65,18 @@ The MPOG analysts observed communication noise in the dialogue between them and |
69 | 65 | their superiors and in the dialogues with the development team that were |
70 | 66 | intermediated by the superiors. They said that direct dialogue with the |
71 | 67 | development team and biweekly visits to the university's lab \textbf{reduce |
72 | -communication misunderstood}: \textit{``At this point, the communication started to | |
73 | -change.. started to improve''}. According to another interviewee, this new | |
74 | -dynamic unified the two sides: \textit{``I believe it was very positive, we also liked to | |
75 | -go there, to interact with the team. I think it brought more unity, more | |
76 | -integration into the project''}. The participation of the MPOG staff was also | |
77 | -considered positive by {72.9\%} of the undegraduates and to {81.1\%} of them | |
78 | -think the presence of MPOG staff in sprint ceremonies was important for the | |
79 | -development. In addition, to \textbf{better meet expectations of both sides} | |
80 | -regarding the requirements developed, {75.6\%} of students believe that writing | |
81 | -the requirements together with the MPOG staff was very important. According to | |
82 | -one of them \textit{``Joint planning and timely meetings were very important for | |
83 | -understanding the needs of MPOG''}. | |
68 | +communication misunderstood}: \textit{``At this point, the communication | |
69 | +started to change.. started to improve''}. According to another interviewee, | |
70 | +this new dynamic unified the two sides: \textit{``I believe it was very | |
71 | +positive, we also liked to go there, to interact with the team. I think it | |
72 | +brought more unity, more integration into the project''}. The participation of | |
73 | +the MPOG staff was also considered positive by {72.9\%} of the undegraduates | |
74 | +and to {81.1\%} of them think the presence of MPOG staff in sprint ceremonies | |
75 | +was important for the development. In addition, to \textbf{better meet | |
76 | +expectations of both sides} regarding the requirements developed, {75.6\%} of | |
77 | +students believe that writing the requirements together with the MPOG staff was | |
78 | +very important. According to one of them \textit{``Joint planning and timely | |
79 | +meetings were very important for understanding the needs of MPOG''}. | |
84 | 80 | |
85 | 81 | An imported consequence of this direct government-academia interaction in |
86 | 82 | laboratory was empathy, as reported by one of the interviewees \textit{``You |
... | ... | @@ -92,27 +88,28 @@ side, we also felt more encouraged to validate faster and give faster feedback |
92 | 88 | to the teams [..] We gave this feedback fast and they also gave quick feedback |
93 | 89 | for any our questions''}. The teams' synchronization was reinforced with the |
94 | 90 | implementation of a Continuous Delivery pipeline. The benefits of this approach |
95 | -were presented in our previous work \cite{siqueira2018cd} and corroborate these research | |
96 | -results. To 81.1\% of students and 75\% of senior developers, deploying new | |
97 | -versions of the SPB portal in production was a motivator during the project. | |
91 | +were presented in our previous work \cite{siqueira2018cd} and corroborate these | |
92 | +research results. To 81.1\% of students and 75\% of senior developers, | |
93 | +deploying new versions of the SPB portal in production was a motivator during | |
94 | +the project. | |
98 | 95 | |
99 | -One of the MPOG analyst interviewed also noted these releases also helped to | |
100 | -\textbf{overcome the government bias regarding low productivity of collaborative | |
101 | -projects with academia}: \textit{``At first, the government staff had a bias that | |
102 | -universities do not deliver. We overcame that bias in the course of the project. | |
103 | -We deliver a lot and with quality. Today, I think if we had paid the same amount | |
104 | -for a company, it would not have done what was delivered and with the quality | |
105 | -that was delivered with the price that was paid''}. Additionally, the deployment | |
106 | -in production of each new version also \textbf{improve the translation of the | |
107 | -process from one side to the other}, as mentioned by MPOG analyst \textit{``We had an | |
108 | -overview at the strategic level. When we went down to the technical level, plan | |
109 | -the release every four months was difficult. But in the end, I think this has | |
110 | -not been a problem. A project you are delivering, the results are going to | |
111 | -production, the code is quality, the team is qualified/capable and the project | |
112 | -is doing well, it does not impact as much in practice''}. | |
96 | +One of the MPOG analyst interviewed also noted these releases also helped to | |
97 | +\textbf{overcome the government bias regarding low productivity of | |
98 | +collaborative projects with academia}: \textit{``At first, the government staff | |
99 | +had a bias that universities do not deliver. We overcame that bias in the | |
100 | +course of the project. We deliver a lot and with quality. Today, I think if we | |
101 | +had paid the same amount for a company, it would not have done what was | |
102 | +delivered and with the quality that was delivered with the price that was | |
103 | +paid''}. Additionally, the deployment in production of each new version also | |
104 | +\textbf{improve the translation of the process from one side to the other}, as | |
105 | +mentioned by MPOG analyst \textit{``We had an overview at the strategic level. | |
106 | +When we went down to the technical level, plan the release every four months | |
107 | +was difficult. But in the end, I think this has not been a problem. A project | |
108 | +you are delivering, the results are going to production, the code is quality, | |
109 | +the team is qualified/capable and the project is doing well, it does not impact | |
110 | +as much in practice''}. | |
113 | 111 | |
114 | -\subsection{Split development team into priority work fronts with IT market | |
115 | -specialists} | |
112 | +\subsection{Split development team into priority work fronts with IT professionals} | |
116 | 113 | |
117 | 114 | Four teams were formed to dedicated to the main development demands of the |
118 | 115 | portal: UX, DevOps, System-of-Systems, and Social Networking. External |
... | ... | @@ -125,8 +122,8 @@ has a good similarity to their previous experiences. Their experience |
125 | 122 | \textbf{helped to reconcile development processes and decision making}, as |
126 | 123 | stated by one of the respondent developers \textit{``I think my main |
127 | 124 | contribution was to have balanced the relations between the MPOG staff and the |
128 | -UnB team''}. {62.5\%} of senior developers believe they have collaborated in the | |
129 | -relationship between the management and development processes of the two | |
125 | +UnB team''}. {62.5\%} of senior developers believe they have collaborated in | |
126 | +the relationship between the management and development processes of the two | |
130 | 127 | institutions and {62.5\%} asserted that helped MPOG staff to more clearly |
131 | 128 | express their requests. {62.5\%} of them did not understand MPOG's project |
132 | 129 | management process and {50\%} believe their project productivity was affected |
... | ... | @@ -137,30 +134,30 @@ confidence in the code''}. |
137 | 134 | |
138 | 135 | In addition, with these professionals was possible to \textbf{transferred |
139 | 136 | knowledge of industry and free software to government and academia}. Working |
140 | -with senior developers was important for all interns during the | |
141 | -project. {91\%} of them also believe that working with professionals was | |
142 | -important for learning. {75\%} of senior developers believe that 'Working in | |
143 | -pairs with a senior' and 62.5\% that 'Participate in joint review tasks' were | |
144 | -the tasks with the involvement of them that most contributed to the evolution | |
145 | -of students in the project. And, in guiding a students, {75\%} believe that | |
146 | -this knowledge was widespread among the others in the team. This acquisition | |
147 | -of knowledge was also noted by the government, which stated \textit{``On the side of | |
148 | -UnB, what we perceived was that the project was very big leap when the | |
149 | -original software developers were hired in the case of Noosfero and Colab, | |
150 | -because they had a guide on how to develop things in the best way and were | |
151 | -able to solve non-trivial problems and quickly''}. | |
137 | +with senior developers was important for all interns during the project. {91\%} | |
138 | +of them also believe that working with professionals was important for | |
139 | +learning. {75\%} of senior developers believe that 'Working in pairs with a | |
140 | +senior' and 62.5\% that 'Participate in joint review tasks' were the tasks with | |
141 | +the involvement of them that most contributed to the evolution of students in | |
142 | +the project. And, in guiding a students, {75\%} believe that this knowledge was | |
143 | +widespread among the others in the team. This acquisition of knowledge was also | |
144 | +noted by the government, which stated \textit{``On the side of UnB, what we | |
145 | +perceived was that the project was very big leap when the original software | |
146 | +developers were hired in the case of Noosfero and Colab, because they had a | |
147 | +guide on how to develop things in the best way and were able to solve | |
148 | +non-trivial problems and quickly''}. | |
152 | 149 | |
153 | -The fronts also gained more autonomy to manage their activities. The role | |
154 | -of ``meta-coach'' was defined among the students, to coordinate the interactions | |
155 | -between teams and coach to coordinate each front. Coaches have become a \textbf{point | |
156 | -of reference for the development process}. {89.1\%} of students said that the | |
157 | -presence of the coach was essential to the running of sprint, and for {87.5\%} | |
158 | -of senior developers coaches was essential for their interaction with the team. | |
159 | -MPOG analysts saw coaches as facilitators for their activities and for | |
160 | -communication with the development team. One of the interviewees said \textit{``I | |
161 | -interacted more with the project coordinator and team coaches''}, \textit{``The reason | |
162 | -for this was that the coaches were more likely to meet the requirements, to | |
163 | -ask questions about requirements, to understand some features. interaction with | |
164 | -leaders than with senior developers. Sometimes the coaches brought the question | |
165 | -to the senior developers''}. | |
150 | +The fronts also gained more autonomy to manage their activities. The role of | |
151 | +``meta-coach'' was defined among the students, to coordinate the interactions | |
152 | +between teams and coach to coordinate each front. Coaches have become a | |
153 | +\textbf{point of reference for the development process}. {89.1\%} of students | |
154 | +said that the presence of the coach was essential to the running of sprint, and | |
155 | +for {87.5\%} of senior developers coaches was essential for their interaction | |
156 | +with the team. MPOG analysts saw coaches as facilitators for their activities | |
157 | +and for communication with the development team. One of the interviewees said | |
158 | +\textit{``I interacted more with the project coordinator and team coaches''}, | |
159 | +\textit{``The reason for this was that the coaches were more likely to meet the | |
160 | +requirements, to ask questions about requirements, to understand some features. | |
161 | +interaction with leaders than with senior developers. Sometimes the coaches | |
162 | +brought the question to the senior developers''}. | |
166 | 163 | ... | ... |