Download as
Browse Code »

Commit e47ee2cfa3e68eff29be6163e156f5ca1cf369eb

Authored by Paulo Meireles
1 parent e59065aa
Exists in master and in 2 other branches oss_2018, rework_case_study

[oss-2018] General review - part 2

Showing 2 changed files with 151 additions and 144 deletions   Show diff stats
oss2018/content/03-methods.tex
  Diff comments   View file @e47ee2c
1 1 \section{Research Design}
2 2 \label{sec:researchdesign}
3 3  
4   -We studied practical alternatives to harmonize the software
5   -project lifecycle when confronting different development processes from crucial
6   -stakeholders. We are interested in the relationship between government and
7   -academia from the project management perspective, without the enforcement of
8   -changing their internal processes. We present two research questions that guided
9   -this work:
  4 +We studied practical alternatives to harmonize the software project lifecycle
  5 +when confronting different development processes from crucial stakeholders. We
  6 +are interested in the relationship between government and academia from the
  7 +project management perspective, without the enforcement of changing their
  8 +internal processes. We present two research questions that guided this work:
10 9  
11 10 \textbf{RQ1. }\textit{How to introduce FLOSS and agile best practices into
12 11 government-academia collaboration projects?}
... ... @@ -14,8 +13,8 @@ government-academia collaboration projects?}
14 13 \textbf{RQ2. }\textit{What practices favor effective team management in
15 14 government-academia collaborative projects?}
16 15  
17   -To answer these questions, we used the case study as research method. We selected
18   -as a case the evolution of the Brazilian Public Software (SPB) portal
  16 +To answer these questions, we used the case study as research method. We
  17 +selected as a case the evolution of the Brazilian Public Software (SPB) portal
19 18 \cite{meirelles2017spb}, a government-academia collaborative project based on
20 19 FLOSS systems. To validate our answers, we covered three different points of
21 20 view: developers, government agent, and data collected from the project
... ... @@ -26,10 +25,11 @@ repository.
26 25 The project to evolve the SPB portal was a partnership between government and
27 26 academia held between 2014 and 2016 \cite{meirelles2017spb}. The old version of
28 27 SPB suffered from maintenance problems and design-reality gaps. In this sense,
29   -The Ministry of Planning (MPOG) decided to join the University of Brasília (UnB) and
30   -the University of São Paulo (USP) to develop a new platform. This platform had
31   -as its primary requirement to be based on existing FLOSS projects and integrate
32   -multiple systems into one, providing the end user with a unified experience.
  28 +The Ministry of Planning (MPOG) decided to join the University of Brasília
  29 +(UnB) and the University of São Paulo (USP) to develop a new platform. This
  30 +platform had as its primary requirement to be based on existing FLOSS projects
  31 +and integrate multiple systems into one, providing the end user with a unified
  32 +experience.
33 33  
34 34 In short, the SPB portal evolved into a Collaborative Development Environment
35 35 (CDE) \cite{booch2003}. It was a novelty in the context of the Brazilian
... ... @@ -40,13 +40,12 @@ system-of-systems framework \cite{meirelles2017spb}.
40 40  
41 41 The development of the platform took place at the Advanced Laboratory of
42 42 Production, Research, and Innovation in Software Engineering (LAPPIS/UnB) and
43   -the FLOSS Competence Center at USP (CCSL/USP),
44   -following the workflow of biweekly sprints and 4-month releases. On the
45   -managerial aspect, at the project beginning, the collaboration management and
46   -strategic discussions happened only once a month, when project leaders and MPOG
47   -directors met in person at the ministry's headquarters.
48   -Table~\ref{tab:gov-academia-diff} summarizes the organizational differences
49   -in both involved sides.
  43 +the FLOSS Competence Center at USP (CCSL/USP), following the workflow of
  44 +biweekly sprints and 4-month releases. On the managerial aspect, at the project
  45 +beginning, the collaboration management and strategic discussions happened only
  46 +once a month, when project leaders and MPOG directors met in person at the
  47 +ministry's headquarters. Table~\ref{tab:gov-academia-diff} summarizes the
  48 +organizational differences in both involved sides.
50 49  
51 50 \vspace*{-.5cm}
52 51  
... ... @@ -54,7 +53,7 @@ in both involved sides.
54 53 \centering
55 54 \def\arraystretch{1.2}
56 55 \setlength\tabcolsep{0.2cm}
57   -\resizebox{\textwidth}{!}{
  56 +\resizebox{\textwidth}{!}{%
58 57 \begin{tabular}{m{4.3cm}!{\color{white}\vrule}m{7cm}!{\color{white}\vrule}m{8cm}}
59 58 \rowcolor[HTML]{c0d6e4}
60 59 \textbf{Collaboration peaces} & \textbf{Academia} & \textbf{Goverment} \\
... ... @@ -68,7 +67,7 @@ in both involved sides.
68 67 \textbf{Workplace} & LAPPIS at UnB and CCSL at USP & MPOG headquarters \\
69 68 \rowcolor[HTML]{fafafa}
70 69 \begin{tabular}[c]{@{}l@{}}\textbf{Management} \textbf{approaches}\end{tabular} & FLOSS practices and Agile values & Traditional approach from RUP, CMMI, and PMBOK \\
71   -\end{tabular}
  70 +\end{tabular}%
72 71 }
73 72 \caption{Differences between academia and government sides.}
74 73 \label{tab:gov-academia-diff}
... ... @@ -88,10 +87,11 @@ these decisions and how they favored the collaboration progress.
88 87  
89 88 \subsection{Survey, Interview and Data Collection}
90 89  
91   -We separated the project team into three groups: undergraduate interns, IT professionals (senior
92   -developers and designers), and MPOG analysts. For the first two we sent online questionnaires,
93   -and for the last one, we conducted 2-hour interviews. Table \ref{survey-table}
94   -presents the details of these processes.
  90 +We separated the project team into three groups: undergraduate interns, IT
  91 +professionals (senior developers and designers), and MPOG analysts. For the
  92 +first two we sent online questionnaires, and for the last one, we conducted
  93 +2-hour interviews. Table \ref{survey-table} presents the details of these
  94 +processes.
95 95  
96 96 \vspace*{-.5cm}
97 97  
... ...
oss2018/content/04-results.tex
  Diff comments   View file @e47ee2c
1 1 \section{Results}
2 2 \label{sec:results}
3 3  
4   -The SPB portal project had two phases according to the traceability of
5   -project management activities. The first one, between January 2014 and March
6   -2015, is non-traceable since only the universities managed the development
7   -activities. The communication between government and academia was, generally, in
8   -private channels, such as professional e-mails, personal meetings, and
9   -telephone calls. Therefore, the quantitative data found for this period
10   -are not conclusive or have little expressiveness, and we do not examine them.
  4 +The SPB portal project had two phases according to the traceability of project
  5 +management activities. The first one, between January 2014 and March 2015, is
  6 +non-traceable since only the universities managed the development activities.
  7 +The communication between government and academia was, generally, in private
  8 +channels, such as professional e-mails, personal meetings, and telephone calls.
  9 +Therefore, the quantitative data found for this period are not conclusive or
  10 +have little expressiveness, and we do not examine them.
11 11  
12 12 The second phase, from April 2015 to the end of the project (June 2016), has
13 13 meaningful data. Much of the management and communication activities were
14 14 recorded and published on online channels and tools. During this period, the
15   -development leaders' employed several FLOSS practices and agile values in the
16   -development process. At the end of the project, the academic team had an
17   -empirical management approach for meeting the government bureaucracies.
  15 +development leaders consolidated several FLOSS practices and agile values
  16 +employed in the development process. At the end of the project, the academic
  17 +team had an empirical management approach for meeting the government
  18 +bureaucracies.
18 19  
19 20 \subsection{Use of the system under development to develop the system itself}
20 21  
21   -Due to the platform features for software development and
22   -social network, the development coordinators decided to use the platform under
23   -construction to develop the system itself. Gradually, in addition to development
24   -activities, government and academia migrated the project management and the
25   -communication between teams to the portal environment.
  22 +Due to the platform features for software development and social network, the
  23 +development coordinators decided to use the platform under construction to
  24 +develop the system itself. Gradually, in addition to development activities,
  25 +government and academia migrated the project management and the communication
  26 +between teams to the portal environment.
26 27  
27   -In short, the wiki feature was used for logging meetings, defining
28   -goals, planning sprints, documenting deployment procedures and user guides. The
29   -issue tracker was used for discussing requirements, monitoring features under
  28 +In short, the wiki feature was used for logging meetings, defining goals,
  29 +planning sprints, documenting deployment procedures and user guides. The issue
  30 +tracker was used for discussing requirements, monitoring features under
30 31 development, requesting and recording changes, and validating the delivered
31   -functionalities. Finally, the mailing list was used for collaborative construction
32   -of requirements, defining schedules, and scheduling meetings between institutions.
  32 +functionalities. Finally, the mailing list was used for collaborative
  33 +construction of requirements, defining schedules, and scheduling meetings
  34 +between institutions.
33 35  
34 36 Our surveys report Mailing list (100\%) and Issue Tracker (62.5\%) as the main
35   -means of interaction between senior developers and interns. Developers and MPOG
36   -staff also interacted mostly via Mailing List (87.5\%) and Issue tracker
37   -(50\%). According to one of the interviewees, this movement made the
  37 +means of interaction between senior developers and interns. The development
  38 +team and MPOG staff also interacted mostly via Mailing List (87.5\%) and Issue
  39 +tracker (50\%). According to one of the interviewees, this movement made the
38 40 \textbf{communication more transparent and efficient}. An MPOG analyst said
39 41 that \textit{``Communicating well goes far beyond the speed. It means enabling
40 42 someone to tell everyone about everything that is happening in the project. We
... ... @@ -45,85 +47,88 @@ see everything''}.
45 47  
46 48 Migrating to the SPB platform also \textbf{easied monitoring of activities and
47 49 increased interactions between developers and public servants}. The data
48   -collected from the repository highlight the frequent use of the platform by both
49   -sides teams. In the last 15 months of the project, 59 different authors opened the
50   -central repository issues, 8 of them were MPOG agents. These issues received comments
51   -from 64 distinct users, 9 of them from MPOG. When we consider the issues with more interactions, those which had ten
52   -comments or more, we notice that the government team also felt comfortable in
53   -using the tool to interact directly with the development team. In a set of 102
54   -active issues, MPOG staff created 43 of them (this represents 42\% of the most active issues).
55   -
56   -For the MPOG analysts, interaction via
57   -repository improved communication. \textit{``There was a big evolution, we
58   -increased our communication via Gitlab''}. Migrating to the platform also led MPOG
59   -staff to \textbf{trust the developed code}: \textit{``Everything was validated.
60   -We tested the functionalities and developed the project on the SPB platform
61   -itself. Hence, the use of the system homologated most of its features. From the
62   -moment we began to use it for developing, this validation was constant. We felt
63   -confident in the code produced''}.
  50 +collected from the repository highlight the frequent use of the platform by
  51 +both sides teams. In the last 15 months of the project, 59 different authors
  52 +opened the central repository issues, 8 of them were MPOG agents. These issues
  53 +received comments from 64 distinct users, 9 of them from MPOG. When we consider
  54 +the issues with more interactions, those which had ten comments or more, we
  55 +notice that the government team also felt comfortable in using the tool to
  56 +interact directly with the development team. In a set of 102 active issues,
  57 +MPOG staff created 43 of them (this represents 42\% of the most active issues).
  58 +
  59 +For the MPOG analysts, interaction via repository improved communication.
  60 +\textit{``There was a big evolution, we increased our communication via
  61 +Gitlab''}. Migrating to the platform also led MPOG staff to \textbf{trust the
  62 +developed code}: \textit{``Everything was validated. We tested the
  63 +functionalities and developed the project on the SPB platform itself. Hence,
  64 +the use of the system homologated most of its features. From the moment we
  65 +began to use it for developing, this validation was constant. We felt confident
  66 +in the code produced''}.
64 67  
65 68 The abovementioned decision also collaborated to meet the government's demand
66 69 for meticulous documentation of the software design and stages of development
67 70 without bureaucratizing or modifying the development process. The usage of the
68 71 platform for project team management conducted \textbf{the organic production
69   -of documentation and records}, as mentioned in one of the MPOG responses: \textit{``It was a great
70   -learning experience. There are many things documented in emails as well as
71   -in the portal itself. We can access the tools at any time and find out how we
72   -develop a solution. We can remember the positive project points''}.
  72 +of documentation and records}, as mentioned in one of the MPOG responses:
  73 +\textit{``It was a great learning experience. There are many things documented
  74 +in emails as well as in the portal itself. We can access the tools at any time
  75 +and find out how we develop a solution. We can remember the positive project
  76 +points''}.
73 77  
74 78 \subsection{Brings together government staff and development team}
75 79  
76   -In the first phase of the project, the interviewed MPOG
77   -analysts did not participate in any direct interaction with any university
78   -representative, even though they were the ones in charge of the government in
79   -ensuring the collaboration agreement and the delivery of the products. Because
80   -of this, they relied on feedback from their superiors on inter-institutional
81   -meetings. They reported that there was significant communication noise in the
82   -internal dialogues with their superiors, as well as between their superiors and
83   -the development team.
84   -
85   -In the second phase of the project, these analysts became direct representatives
86   -of the government and started to visit the university's laboratory bi-weekly.
87   -One of the analysts believed that
88   -\textit{``at this point, the communication started to change''}. The new
89   -dynamics \textit{reduced communication misunderstandings and unified both
90   -sides}, as reported by another interviewee: \textit{``It was very positive. We
91   -liked to go there and to interact with the team. I think it brought more unity,
92   -more integration into the project''}. {73\%} of the interns considered positive
93   -the direct participation of the MPOG staff, and {81\%} of them believed the
94   -presence of government staff in sprint ceremonies was relevant for the project
95   -development. For 76\% of the interns, writing the requirements together with the
96   -MPOG staff was very important to \textbf{better meet expectations of both
97   -sides}. According to one of them, \textit{``Joint planning and timely meetings
98   -were very important for understanding the needs of MPOG''}.
  80 +In the first phase of the project, the interviewed MPOG analysts did not
  81 +participate in any direct interaction with any university representative, even
  82 +though they were the ones in charge of the government in ensuring the
  83 +collaboration agreement and the delivery of the products. Because of this, they
  84 +relied on feedback from their superiors on inter-institutional meetings. They
  85 +reported that there was significant communication noise in the internal
  86 +dialogues with their superiors, as well as between their superiors and the
  87 +development team.
  88 +
  89 +In the second phase of the project, these analysts became direct
  90 +representatives of the government and started to visit the university's
  91 +laboratory bi-weekly. One of the analysts believed that \textit{``at this
  92 +point, the communication started to change''}. The new dynamics \textbf{reduced
  93 +communication misunderstandings and unified both sides}, as reported by another
  94 +interviewee: \textit{``It was very positive. We liked to go there and to
  95 +interact with the team. I think it brought more unity, more integration into
  96 +the project''}. {73\%} of the interns considered positive the direct
  97 +participation of the MPOG staff, and {81\%} of them believed the presence of
  98 +government staff in sprint ceremonies was relevant for the project development.
  99 +For 76\% of the interns, writing the requirements together with the MPOG staff
  100 +was very important to \textbf{better meet expectations of both sides}.
  101 +According to one of them, \textit{``Joint planning and timely meetings were
  102 +very important for understanding the needs of MPOG''}.
99 103  
100 104 The closest dialogue between government and academia generated empathy, as
101 105 reported by one of the interviewees: \textit{``Knowing people in person makes a
102 106 big difference in the relationship because it causes empathy. You know who that
103   -person is. He's not merly a name''}. Consequently, this empathy helped to \textbf{synchronize
104   -the execution pace of activities}: \textit{``Visiting the lab and meeting
105   -the developers encouraged us to validate resources faster and give faster feedback to
106   -the team. In return, they also quickly answered us any question''}.
  107 +person is. He's not merly a name''}. Consequently, this empathy helped to
  108 +\textbf{synchronize the execution pace of activities}: \textit{``Visiting the
  109 +lab and meeting the developers encouraged us to validate resources faster and
  110 +give faster feedback to the team. In return, they also quickly answered us any
  111 +question''}.
107 112  
108 113 The implementation of a Continuous Delivery pipeline also reinforced the teams'
109   -synchronization \cite{siqueira2018cd} . For 81\% of the interns and 75\% of the senior
110   -developers, deploying new versions of the SPB portal in production was a
111   -motivator during the project. On the government side, this approach helped to
112   -\textbf{overcome the government bias toward low productivity of
  114 +synchronization \cite{siqueira2018cd} . For 81\% of the interns and 75\% of
  115 +the IT professionals, deploying new versions of the SPB portal in production
  116 +was a motivator during the project. On the government side, this approach
  117 +helped to \textbf{overcome the government bias toward low productivity of
113 118 collaborative projects with academia}, as mentioned by themselves:
114 119 \textit{``Government staff has a bias that universities do not deliver
115 120 products. However, in this project, we made many deliveries with high quality.
116 121 Nowadays, I think if we had paid the same amount for a company, it would not
117   -have done the amount of features we did with the technical quality we have''}. Additionally, the
118   -deployment of each new version also \textbf{share a common understanding of the
119   -process from one side to the other}, as mentioned by a MPOG analyst: \textit{``We
120   -had only the strategic vision of the project. When we needed to deal with
121   -technical issues, we had some difficulty planning the four-month releases.
122   -However, in the last stages of the project I realized that this was not a
123   -problem. The team was delivering and the results were available in production.
124   -The team was qualified, the code had quality, and the project was well executed.
125   -So in practice, our difficulty in interpreting the technical details did not
126   -impact the release planning''}.
  122 +have done the amount of features we did with the technical quality we have''}.
  123 +Additionally, the deployment of each new version also \textbf{share a common
  124 +understanding of the process from one side to the other}, as mentioned by a
  125 +MPOG analyst: \textit{``We had only the strategic vision of the project. When
  126 +we needed to deal with technical issues, we had some difficulty planning the
  127 +four-month releases. However, in the last stages of the project I realized
  128 +that this was not a problem. The team was delivering and the results were
  129 +available in production. The team was qualified, the code had quality, and the
  130 +project was well executed. So in practice, our difficulty in interpreting the
  131 +technical details did not impact the release planning''}.
127 132  
128 133 \subsection{Organized development team into priority fronts, and for each one, hire at least one specialist from the IT market}
129 134  
... ... @@ -138,44 +143,46 @@ The presence of senior developers in the project contributed to
138 143 \textbf{conciliate the development processes of each institution and make
139 144 better technical decisions}, as quoted in one of the answers to the senior
140 145 developer's questionnaire: \textit{``I think my main contribution was to
141   -balance the relations between the MPOG staff and the university team''}. {63\%} of
142   -the senior developers believed they have collaborated to conciliate the management
143   -and development process between the two institutions and also {63\%} of them
144   -helped MPOG staff express their requests more clearly. Government
  146 +balance the relations between the MPOG staff and the university team''}. {63\%}
  147 +of the IT professionals believed they have collaborated to conciliate the
  148 +management and development process between the two institutions and also {63\%}
  149 +of them helped MPOG staff express their requests more clearly. Government
145 150 analysts were also more open to suggestions from these developers:
146   -\textit{``They are upstream developers of the systems that
147   -integrate the platform. They conveyed trust, and then we trust in the developed
148   -code''}. According to questionnaire responses, senior developers largely agreed with the
  151 +\textit{``They are upstream developers of the systems that integrate the
  152 +platform. They conveyed trust, and then we trust in the developed code''}.
  153 +According to questionnaire responses, IT professionals largely agreed with the
149 154 project development process. For 63\%, this process has close similarity to
150 155 their previous experiences. In contrast, {62.5\%} of them did not understand
151 156 the MPOG's project management process and {50\%} believed this process could
152 157 affect their project productivity.
153 158  
154   -The senior developers were also responsible for \textbf{improving the management
155   -and technical knowledge} of the interns about practices from industry and open
156   -source projects. {91\%} of the interns believed that working with professionals
157   -was essential for learning, and, for all of them, working with senior developers
158   -was important during the project. {75\%} of the senior developers believed that ``Working
159   -in pairs with a senior'' and 63\% that ``Participate in joint review tasks''
160   -were the tasks with the involvement of them that most contributed to the
161   -evolution of university interns in the project. {75\%} believed that the knowledge
162   -shared by them to one intern was widespread among the others in the team.
163   -Government analysts also pointed this knowledge sharing: \textit{``On
164   -the university side, we noticed a significant improvement in the platform
165   -with the hiring of the systems original developers. They had a guide on
166   -how to best develop each feature and were able to solve non-trivial problems
167   -quickly''}.
168   -
169   -Organizing the development team and hiring senior developers allowed each team to
170   -\textbf{self-organize and gain more autonomy in the management of their tasks}.
171   -There was a development coach to lead each team, and a ``meta-coach'' supported
172   -all of them in their internal management activities. The coaches (most advanced
173   -university interns) were points of reference in the development process. {89\%} of the
174   -interns said that the presence of the coach was essential to the sprint's
175   -running, and for {88\%} of the senior developers coaches was essential for their
176   -interaction with the team. MPOG analysts saw coaches as facilitators their
177   -activities and communication with the development team. They said \textit{``I
178   -interacted more with the project coordinator (professor) and team coaches''},
179   -\textit{``Usually, we contact a coach to clarify some requirements or to
180   -understand some feature. The coaches were more available than senior
181   -developers and, sometimes, they would take our question to a senior developer''}.
  159 +The senior developers were also responsible for \textbf{improving the
  160 +management and technical knowledge} of the interns about practices from
  161 +industry and open source projects. {91\%} of the interns believed that working
  162 +with professionals was essential for learning, and, for all of them, working
  163 +with IT professionals was important during the project. {75\%} of the IT
  164 +professionals believed that ``Working in pairs with a senior'' and 63\% that
  165 +``Participate in joint review tasks'' were the tasks with the involvement of
  166 +them that most contributed to the evolution of the interns in the project.
  167 +{75\%} believed that the knowledge shared by them to one intern was widespread
  168 +among the others in the team. Government analysts also pointed this knowledge
  169 +sharing: \textit{``On the university side, we noticed a significant improvement
  170 +in the platform with the hiring of the systems original developers. They had a
  171 +guide on how to best develop each feature and were able to solve non-trivial
  172 +problems quickly''}.
  173 +
  174 +Organizing the development team and hiring of the IT professionals allowed each
  175 +team to \textbf{self-organize and gain more autonomy in the management of their
  176 +tasks}. There was a development coach to lead each team, and a ``meta-coach''
  177 +supported all of them in their internal management activities. The coaches
  178 +(most advanced interns) were points of reference in the development process.
  179 +{89\%} of the interns said that the presence of the coach was essential to the
  180 +sprint's running, and for {88\%} of the of the IT professionals the coaches was
  181 +essential for their interaction with the development team. MPOG analysts saw
  182 +the coaches as facilitators their activities and communication with the
  183 +development team. They said \textit{``I interacted more with the project
  184 +coordinator (professor) and team coaches (interns)''}, \textit{``Usually, we
  185 +contact a coach to clarify some requirements or to understand some feature. The
  186 +coaches were more available than senior developers and, sometimes, they would
  187 +take our question to a senior developer''}.
  188 +
... ...