Download as
Browse Code »

Commit 892b9b0481e73d4bfa6ba7699d20b2d3b5b039ac

Authored by Paulo Meireles
1 parent fef7abc2
Exists in master and in 3 other branches oss_2018, oss_2018_siqueira, rework_case_study

[oss-2018] Reviewing Results

Showing 1 changed file with 108 additions and 111 deletions   Show diff stats
icse2018/content/05-results.tex
  Diff comments   View file @892b9b0
1 1 \section{Results}
2 2 \label{sec:results}
3 3  
4   -%TODO: Talvez esse paragráfo tem que está no Research Design
5   -%%
6   -The case study was analyzed and divided into two phases according to the project
7   -management model. In the second phase (after one year of execution), several
8   -practices have been applied to harmonize the cultural and organizational
9   -divergences of the institutions involved.
10   -%%
11   -At the end of the project, an empirical
12   -model of management and development process was created by aligning experiences
13   -from the FLOSS universe, academic research and bureaucracies needed by the
14   -government. In this section, we present by context the practices adopted in this
15   -second phase and show the benefits generated by its deployment.
  4 +The case study had two phases according to the project management model. In the
  5 +second phase (about one year of execution), several practices have been applied
  6 +to harmonize the cultural and organizational divergences of the institutions
  7 +involved. At the end of the project, an empirical model of management and
  8 +development process was stabilized by aligning experiences from the open source
  9 +ecosystem, academic research, and bureaucracies needed by the government. In
  10 +this section, we present by context the practices adopted in this second phase
  11 +and show the benefits generated by its deployment.
16 12  
17 13 \subsection{Project management and communication on the developing platform
18 14 itself}
19 15  
20   -After nine months of project activities, the first version of new SPB Portal was
21   -released. From this point, we started to migrate the management and
22   -communication interactions to the platform under development. In short, Wiki
23   -feature was used for meeting logging, defining goals, sprint planning, and
24   -documentation of deployment processes and administration resources guide. Issue
25   -tracker was used for discussing requirements, monitoring the features under
26   -development, registering changes, and validating functionalities delivered. Finally, the
27   -whole team used Mailing list to defining schedules of meetings and deliveries
28   -and also to collaborative definition of requirements.
  16 +After nine months of project activities, the first version of new SPB Portal
  17 +was released. From this point, we started to migrate the management and
  18 +communication interactions to the platform under development. In short, Wiki
  19 +feature was used for meeting logging, defining goals, sprint planning, and
  20 +documentation of deployment processes and administration resources guide. Issue
  21 +tracker was used for discussing requirements, monitoring the features under
  22 +development, registering changes, and validating functionalities delivered.
  23 +Finally, the whole team used Mailing list to defining schedules of meetings and
  24 +deliveries and also to collaborative definition of requirements.
29 25  
30   -Our surveys reports Mailing list (100\%) and Issue Tracker (62.5\%) as the main means
31   -of interaction between senior developers and undergraduates. Developers and MPOG
32   -staff also interacted mostly via Mailing List (87.5\%) and Issue tracker (50\%).
33   -According to research findings, this movement made \textbf{communication more
34   -transparent and efficient}. A MPOG IT analyst said that the
35   -\textit{``Communicating well goes far beyond the speed, it is someone being able
36   -to communicate to everyone everything that is happening in the project. We did
37   -not use emails. We use more mailing list and avoid e-mails. It helped a lot
38   -because everything was public and did not pollute our mailbox. You wanted to
39   -know something, could go there and look at what was happening''}.
  26 +Our surveys reports Mailing list (100\%) and Issue Tracker (62.5\%) as the main
  27 +means of interaction between senior developers and undergraduates. Developers
  28 +and MPOG staff also interacted mostly via Mailing List (87.5\%) and Issue
  29 +tracker (50\%). According to research findings, this movement made
  30 +\textbf{communication more transparent and efficient}. A MPOG IT analyst said
  31 +that the \textit{``Communicating well goes far beyond the speed, it is someone
  32 +being able to communicate to everyone everything that is happening in the
  33 +project. We did not use emails. We use more mailing list and avoid e-mails. It
  34 +helped a lot because everything was public and did not pollute our mailbox. You
  35 +wanted to know something, could go there and look at what was happening''}.
40 36  
41   -Migrating to SPB platform also provided an \textbf{easier monitoring and
42   -increase interactions between development team and public servants by
43   -coordinators}. As shown by collected data, in the last 15 months of the project,
44   -the issues have 59 different authors (8 from MPOG
45   -staff), and commented by 64 different users (9 from MPOG staff and users).
46   -Considering issues with higher level of interaction those that have 10 or more
47   -comments, in a set of 102 issues, MPOG staff authored 43 issues (which represents
48   -42\% of these most active issues). A MPOG analyst highlighted that
49   -\textit{``there was a lot of evolution, a lot of communication via Gitlab''}.
50   -This interaction also led MPOG staff to \textbf{trust developed code}:
51   -\textit{``Everything was validated, we tested the features and the project was
52   -developed inside the platform, so that the feature was validated in the
53   -development of the software itself. From the moment we installed it, and
54   -began to use it for development, this validation was constant. We felt confident
55   -in the features''}.
  37 +Migrating to SPB platform also provided an \textbf{easier monitoring and
  38 +increase interactions between development team and public servants by
  39 +coordinators}. As shown by collected data, in the last 15 months of the
  40 +project, the issues have 59 different authors (8 from MPOG staff), and
  41 +commented by 64 different users (9 from MPOG staff and users). Considering
  42 +issues with higher level of interaction those that have 10 or more comments, in
  43 +a set of 102 issues, MPOG staff authored 43 issues (which represents 42\% of
  44 +these most active issues). A MPOG analyst highlighted that \textit{``there was
  45 +a lot of evolution, a lot of communication via Gitlab''}. This interaction
  46 +also led MPOG staff to \textbf{trust developed code}: \textit{``Everything was
  47 +validated, we tested the features and the project was developed inside the
  48 +platform, so that the feature was validated in the development of the software
  49 +itself. From the moment we installed it, and began to use it for development,
  50 +this validation was constant. We felt confident in the features''}.
56 51  
57   -One of the main concerns of traditional approach is meticulous documentation of
  52 +One of the main concerns of traditional approach is meticulous documentation of
58 53 the software designed and the development steps. With this aforementioned
59   -decision, we could meet this government demand without bureaucracies and changes
60   -in our development process, \textbf{producting organically documentation and
61   -records} in the platform itself, as one of the MPOG response evidenced:
62   -\textit{``For me, it was a lot of learning. There is a lot of things documented
63   -in the e-mails and also in the portal itself. At any moment we can go there and
64   -see how it worked, how someone did something. We can recover those good points''}.
  54 +decision, we could meet this government demand without bureaucracies and
  55 +changes in our development process, \textbf{producting organically
  56 +documentation and records} in the platform itself, as one of the MPOG response
  57 +evidenced: \textit{``For me, it was a lot of learning. There is a lot of things
  58 +documented in the e-mails and also in the portal itself. At any moment we can
  59 +go there and see how it worked, how someone did something. We can recover those
  60 +good points''}.
65 61  
66 62 \subsection{Bringing together government staff and development team}
67 63  
... ... @@ -69,18 +65,18 @@ The MPOG analysts observed communication noise in the dialogue between them and
69 65 their superiors and in the dialogues with the development team that were
70 66 intermediated by the superiors. They said that direct dialogue with the
71 67 development team and biweekly visits to the university's lab \textbf{reduce
72   -communication misunderstood}: \textit{``At this point, the communication started to
73   -change.. started to improve''}. According to another interviewee, this new
74   -dynamic unified the two sides: \textit{``I believe it was very positive, we also liked to
75   -go there, to interact with the team. I think it brought more unity, more
76   -integration into the project''}. The participation of the MPOG staff was also
77   -considered positive by {72.9\%} of the undegraduates and to {81.1\%} of them
78   -think the presence of MPOG staff in sprint ceremonies was important for the
79   -development. In addition, to \textbf{better meet expectations of both sides}
80   -regarding the requirements developed, {75.6\%} of students believe that writing
81   -the requirements together with the MPOG staff was very important. According to
82   -one of them \textit{``Joint planning and timely meetings were very important for
83   -understanding the needs of MPOG''}.
  68 +communication misunderstood}: \textit{``At this point, the communication
  69 +started to change.. started to improve''}. According to another interviewee,
  70 +this new dynamic unified the two sides: \textit{``I believe it was very
  71 +positive, we also liked to go there, to interact with the team. I think it
  72 +brought more unity, more integration into the project''}. The participation of
  73 +the MPOG staff was also considered positive by {72.9\%} of the undegraduates
  74 +and to {81.1\%} of them think the presence of MPOG staff in sprint ceremonies
  75 +was important for the development. In addition, to \textbf{better meet
  76 +expectations of both sides} regarding the requirements developed, {75.6\%} of
  77 +students believe that writing the requirements together with the MPOG staff was
  78 +very important. According to one of them \textit{``Joint planning and timely
  79 +meetings were very important for understanding the needs of MPOG''}.
84 80  
85 81 An imported consequence of this direct government-academia interaction in
86 82 laboratory was empathy, as reported by one of the interviewees \textit{``You
... ... @@ -92,27 +88,28 @@ side, we also felt more encouraged to validate faster and give faster feedback
92 88 to the teams [..] We gave this feedback fast and they also gave quick feedback
93 89 for any our questions''}. The teams' synchronization was reinforced with the
94 90 implementation of a Continuous Delivery pipeline. The benefits of this approach
95   -were presented in our previous work \cite{siqueira2018cd} and corroborate these research
96   -results. To 81.1\% of students and 75\% of senior developers, deploying new
97   -versions of the SPB portal in production was a motivator during the project.
  91 +were presented in our previous work \cite{siqueira2018cd} and corroborate these
  92 +research results. To 81.1\% of students and 75\% of senior developers,
  93 +deploying new versions of the SPB portal in production was a motivator during
  94 +the project.
98 95  
99   -One of the MPOG analyst interviewed also noted these releases also helped to
100   -\textbf{overcome the government bias regarding low productivity of collaborative
101   -projects with academia}: \textit{``At first, the government staff had a bias that
102   -universities do not deliver. We overcame that bias in the course of the project.
103   -We deliver a lot and with quality. Today, I think if we had paid the same amount
104   -for a company, it would not have done what was delivered and with the quality
105   -that was delivered with the price that was paid''}. Additionally, the deployment
106   -in production of each new version also \textbf{improve the translation of the
107   -process from one side to the other}, as mentioned by MPOG analyst \textit{``We had an
108   -overview at the strategic level. When we went down to the technical level, plan
109   -the release every four months was difficult. But in the end, I think this has
110   -not been a problem. A project you are delivering, the results are going to
111   -production, the code is quality, the team is qualified/capable and the project
112   -is doing well, it does not impact as much in practice''}.
  96 +One of the MPOG analyst interviewed also noted these releases also helped to
  97 +\textbf{overcome the government bias regarding low productivity of
  98 +collaborative projects with academia}: \textit{``At first, the government staff
  99 +had a bias that universities do not deliver. We overcame that bias in the
  100 +course of the project. We deliver a lot and with quality. Today, I think if we
  101 +had paid the same amount for a company, it would not have done what was
  102 +delivered and with the quality that was delivered with the price that was
  103 +paid''}. Additionally, the deployment in production of each new version also
  104 +\textbf{improve the translation of the process from one side to the other}, as
  105 +mentioned by MPOG analyst \textit{``We had an overview at the strategic level.
  106 +When we went down to the technical level, plan the release every four months
  107 +was difficult. But in the end, I think this has not been a problem. A project
  108 +you are delivering, the results are going to production, the code is quality,
  109 +the team is qualified/capable and the project is doing well, it does not impact
  110 +as much in practice''}.
113 111  
114   -\subsection{Split development team into priority work fronts with IT market
115   -specialists}
  112 +\subsection{Split development team into priority work fronts with IT professionals}
116 113  
117 114 Four teams were formed to dedicated to the main development demands of the
118 115 portal: UX, DevOps, System-of-Systems, and Social Networking. External
... ... @@ -125,8 +122,8 @@ has a good similarity to their previous experiences. Their experience
125 122 \textbf{helped to reconcile development processes and decision making}, as
126 123 stated by one of the respondent developers \textit{``I think my main
127 124 contribution was to have balanced the relations between the MPOG staff and the
128   -UnB team''}. {62.5\%} of senior developers believe they have collaborated in the
129   -relationship between the management and development processes of the two
  125 +UnB team''}. {62.5\%} of senior developers believe they have collaborated in
  126 +the relationship between the management and development processes of the two
130 127 institutions and {62.5\%} asserted that helped MPOG staff to more clearly
131 128 express their requests. {62.5\%} of them did not understand MPOG's project
132 129 management process and {50\%} believe their project productivity was affected
... ... @@ -137,30 +134,30 @@ confidence in the code''}.
137 134  
138 135 In addition, with these professionals was possible to \textbf{transferred
139 136 knowledge of industry and free software to government and academia}. Working
140   -with senior developers was important for all interns during the
141   -project. {91\%} of them also believe that working with professionals was
142   -important for learning. {75\%} of senior developers believe that 'Working in
143   -pairs with a senior' and 62.5\% that 'Participate in joint review tasks' were
144   -the tasks with the involvement of them that most contributed to the evolution
145   -of students in the project. And, in guiding a students, {75\%} believe that
146   -this knowledge was widespread among the others in the team. This acquisition
147   -of knowledge was also noted by the government, which stated \textit{``On the side of
148   -UnB, what we perceived was that the project was very big leap when the
149   -original software developers were hired in the case of Noosfero and Colab,
150   -because they had a guide on how to develop things in the best way and were
151   -able to solve non-trivial problems and quickly''}.
  137 +with senior developers was important for all interns during the project. {91\%}
  138 +of them also believe that working with professionals was important for
  139 +learning. {75\%} of senior developers believe that 'Working in pairs with a
  140 +senior' and 62.5\% that 'Participate in joint review tasks' were the tasks with
  141 +the involvement of them that most contributed to the evolution of students in
  142 +the project. And, in guiding a students, {75\%} believe that this knowledge was
  143 +widespread among the others in the team. This acquisition of knowledge was also
  144 +noted by the government, which stated \textit{``On the side of UnB, what we
  145 +perceived was that the project was very big leap when the original software
  146 +developers were hired in the case of Noosfero and Colab, because they had a
  147 +guide on how to develop things in the best way and were able to solve
  148 +non-trivial problems and quickly''}.
152 149  
153   -The fronts also gained more autonomy to manage their activities. The role
154   -of ``meta-coach'' was defined among the students, to coordinate the interactions
155   -between teams and coach to coordinate each front. Coaches have become a \textbf{point
156   -of reference for the development process}. {89.1\%} of students said that the
157   -presence of the coach was essential to the running of sprint, and for {87.5\%}
158   -of senior developers coaches was essential for their interaction with the team.
159   -MPOG analysts saw coaches as facilitators for their activities and for
160   -communication with the development team. One of the interviewees said \textit{``I
161   -interacted more with the project coordinator and team coaches''}, \textit{``The reason
162   -for this was that the coaches were more likely to meet the requirements, to
163   -ask questions about requirements, to understand some features. interaction with
164   -leaders than with senior developers. Sometimes the coaches brought the question
165   -to the senior developers''}.
  150 +The fronts also gained more autonomy to manage their activities. The role of
  151 +``meta-coach'' was defined among the students, to coordinate the interactions
  152 +between teams and coach to coordinate each front. Coaches have become a
  153 +\textbf{point of reference for the development process}. {89.1\%} of students
  154 +said that the presence of the coach was essential to the running of sprint, and
  155 +for {87.5\%} of senior developers coaches was essential for their interaction
  156 +with the team. MPOG analysts saw coaches as facilitators for their activities
  157 +and for communication with the development team. One of the interviewees said
  158 +\textit{``I interacted more with the project coordinator and team coaches''},
  159 +\textit{``The reason for this was that the coaches were more likely to meet the
  160 +requirements, to ask questions about requirements, to understand some features.
  161 +interaction with leaders than with senior developers. Sometimes the coaches
  162 +brought the question to the senior developers''}.
166 163  
... ...