[oss-2018] Reviewing the Results section

Paulo Meireles
1 parent 5e6778c2
Showing 1 changed file with 141 additions and 142 deletions Show diff stats
oss2018/content/04-results.tex
 \section{Results}
 \label{sec:results}
  
-We divided the case study into two phases, according to the traceability of the
-project management activities. We consider the first phase, between January
-2014 and March 2015, as non-traceable. In this period, only UnB managed the
-development activities. The inter-institutional monitoring of the project was
-reduced to the definition of strategic goals in meetings between coordinators,
-and the communication between government and academia was dispersed in private
-channels, such as professional e-mails, personal meetings, and telephone calls.
-Because of this, the quantitative data found for this period are inconclusive or
-have little expressiveness, and we did not examine them.
+The SPB portal project had two phases according to the traceability of the
+project management activities. The first one, between January 2014 and March
+2015, is non-traceable. In this period, only UnB managed the development
+activities. Professors and MPOG coordinators had meetings to define strategic
+goals. The communication between government and academia was, generally, in
+private channels, such as professional e-mails, personal meetings, and
+telephone calls. Because of this, the quantitative data found for this period
+are inconclusive or have little expressiveness, and we did not examine them.
  
 The second phase, from April 2015 to the end of the project (June 2016), has a
 more considerable wealth of data. Much of the management and communication
 activities were recorded and published on online channels and tools. During
-this period, several open source practices were applied to the development
-process to harmonize the cultural and organizational divergences of the
-institutions involved. At the end of the project, an empirical model of
-communication and management was built using experiments in free software
-ecosystems to cater to government bureaucracies.
-
-In this section, we list the macro-decisions taken intuitively during the
-project and the practices that made these decisions concrete. We use data
-collected from the main repository to map best practices and, with the
-respondents' answers, we analyzed how each decision benefited the project
-collaboration.
+this period, several FLOSS practices and agile values were applied to the
+development process to harmonize the cultural and organizational divergences of
+the institutions involved. At the end of the project, an empirical approach to
+communication and management was built using the development leaders'
+experiences in FLOSS and agile projects to cater to government bureaucracies.
+
+In this section, we list the macro-decisions taken during the project and the
+practices that made these decisions concrete. We use data collected from the
+central repository to map best practices and, with the respondents' answers, we
+analyzed how each decision benefited the project collaboration.
+
  
 \subsection{Use of system under development to develop the system itself}
  
-The first version of the new SPB portal was released in production nine months
-after the project beginning. Due to the platform features for software
-development and social network, the UnB coordinators decided to use the system
-under construction to develop the system itself. Gradually, in addition to
+UnB team released the first version of the new SPB portal nine months after the
+project beginning. Due to the platform features for software development and
+social network, the UnB coordinators decided to use the platform under
+construction to develop the system itself. Gradually, in addition to
 development activities, government and academia migrated the project management
-and the communication between teams to the portal environment. In short, the wiki
-feature was used for logging meetings, defining goals, planning sprints,
+and the communication between teams to the portal environment. In short, the
+wiki feature was used for logging meetings, defining goals, planning sprints,
 documenting deployment procedures and user guides. The issue tracker was used
 for discussing requirements, monitoring features under development, requesting
-and recording changes, and validating the delivered funcionalities. Finally, the
-mailing list was used by the entire team for collaborative construction of
+and recording changes, and validating the delivered functionalities. Finally,
+the mailing list was used by the entire team for collaborative construction of
 requirements, defining schedules, and scheduling meetings between institutions.
  
 Our surveys report Mailing list (100\%) and Issue Tracker (62.5\%) as the main
-means of interaction between senior developers and interns. Developers
-and MPOG staff also interacted mostly via Mailing List (87.5\%) and Issue
-tracker (50\%).  According to one of the interviewees, this movement made the
+means of interaction between senior developers and interns. Developers and MPOG
+staff also interacted mostly via Mailing List (87.5\%) and Issue tracker
+(50\%).  According to one of the interviewees, this movement made the
 \textbf{communication more transparent and efficient}. An MPOG analyst said
 that \textit{``Communicating well goes far beyond the speed. It means enabling
 someone to tell everyone about everything that is happening in the project. We
-did not use emails, we use more mailing list and avoid emails. This usage helped
-us a lot because everything was public and did not pollute our email box. So,
-when you wanted to know something, you could access the SPB list to see
-everything that was happening''}.
-
-Migrating to the SPB platform also \textbf{easied monitoring of
-activities and increased interactions between developers and public servants}.
-The data collected from the repository evidence the frequent use of the platform
-by the academic team and the government team. In the last 15 months of the
-project, the main repository issues were opened by 59 different authors, 8 of them
-MPOG agents. These issues received comments from 64 distinct users, 9 of them
-from MPOG. When we consider the issues with much interaction, those who had ten
-comments or more, we notice that the government team also felt comfortable
-with using the tool to interact directly with the development team. In a
-set of 102 issues with much interaction, MPOG staff created 43 of them (this
-represents 42\% of the most active issues). For the MPOG analysts, interaction
-via repository improved communication. \textit{``There was a lot of evolution, a
-lot of communication via Gitlab''}.  Migrating to the platform also led MPOG
+did not use emails, we use more mailing list and avoid emails. This usage
+helped us a lot because everything was public and did not pollute our email
+box. So, when you wanted to know something, you could access the SPB list to
+see everything that was happening.''}.
+
+Migrating to the SPB platform also \textbf{easied monitoring of activities and
+increased interactions between developers and public servants}. The data
+collected from the repository evidence the frequent use of the platform by the
+academic and the government teams. In the last 15 months of the project, the
+central repository issues were opened by 59 different authors, 8 of them MPOG
+agents. These issues received comments from 64 distinct users, 9 of them from
+MPOG. When we consider the issues with much interaction, those who had ten
+comments or more, we notice that the government team also felt comfortable with
+using the tool to interact directly with the development team. In a set of 102
+issues with much interaction, MPOG staff created 43 of them (this represents
+42\% of the most active issues). For the MPOG analysts, interaction via
+repository improved communication. \textit{``There was a lot of evolution, a
+lot of communication via Gitlab.''}.  Migrating to the platform also led MPOG
 staff to \textbf{trust in developed code}: \textit{``Everything was validated.
-We tested the functionalities and developed the project on the platform itself.
-Consequently, all features were checked according to the use of the system.
-From the moment we began to use it for development, this validation was constant.
-We felt confident in the code developed.'}.
+We tested the functionalities and developed the project on the SPB platform
+itself.  Consequently, the use of the system validated the most of features.
+From the moment we began to use it for development, this validation was
+constant. We felt confident in the code developed.''}.
  
 The abovementioned decision also collaborated to meet the government's demand
 for meticulous documentation of the software design and stages of development
-without bureaucratizing or modifying the development process. The team starts to
-\textbf{produce documentation and records organically} on the platform itself, as
-mentioned at one of the MPOG response.: \textit{``For me, it was a great learning
-experience. There are a lot of things documented in emails as well as in the
-portal itself. When necessary, we can access the tools and find out how we
-develop a solution. We can recover these positive points.''}.
+without bureaucratizing or modifying the development process. The team starts
+to \textbf{produce documentation and records organically} on the platform
+itself, as mentioned at one of the MPOG response.: \textit{``For me, it was a
+great learning experience. There are a lot of things documented in emails as
+well as in the portal itself. When necessary, we can access the tools and find
+out how we develop a solution. We can recover these positive points.''}.
  
  
 \subsection{Bring together government staff and development team}
@@ -95,101 +94,101 @@ their superiors, as well as between their superiors and the development team.
 In the second phase of the project, these analysts came to represent the
 government directly in the dialogues with the academia, and they started to
 visit bi-weekly the university's laboratory. One of the analysts believes that
-\textit{``at this point, the communication started to change.''} The new dynamic
-\textit{reduced communication misunderstandings and unified the two sides}, as reported
-by another interviewee: \textit{``It was very positive. We liked to go there and
-to interact with the team. I think it brought more unity, more integration into
-the project''}. {73\%} of the interns consider positive the direct
-participation of the MPOG staff, and {81\%} of them think the presence of
-goverment staff in sprint ceremonies was relevant for the project development.
-For 76\% of interns, writing the requirements together with the MPOG staff was
-very important to \textbf{better meet expectations of both sides}. According to
-one of them \textit{``Joint planning and timely meetings were very important for
-understanding the needs of MPOG''}.
+\textit{``at this point, the communication started to change.''}. The new
+dynamic \textit{reduced communication misunderstandings and unified the two
+sides}, as reported by another interviewee: \textit{``It was very positive. We
+liked to go there and to interact with the team. I think it brought more unity,
+more integration into the project.''}. {73\%} of the interns consider positive
+the direct participation of the MPOG staff, and {81\%} of them think the
+presence of goverment staff in sprint ceremonies was relevant for the project
+development. For 76\% of interns, writing the requirements together with the
+MPOG staff was very important to \textbf{better meet expectations of both
+sides}. According to one of them \textit{``Joint planning and timely meetings
+were very important for understanding the needs of MPOG.''}.
  
 The closest dialogue between government and academia generated empathy, as
 reported by one of the interviewees: \textit{``Knowing people in person makes a
-big difference in the relationship because it causes empathy. You know who is
-that person, it's not simply a name.''}. This subjectively helped to
-\textbf{synchronize the execution pace of activities}, \textit{``When we visited
-the lab and met the team, we realized that this encouraged us to validate
-resources faster and give faster feedback to the team. In return, they also
-quickly answered us any question''}.
-
-The teams' synchronization was reinforced with the implementation of a
-Continuous Delivery pipeline. The benefits of this approach were presented in
-our previous work \cite{siqueira2018cd} and corroborate these research results.
-For 81\% of interns and 75\% of senior developers, deploying new versions of the
-SPB portal in production was a motivator during the project. On the government
-side, this approach helped to \textbf{overcome the government bias regarding the
-low productivity of collaborative projects with academia}, as mentioned by
-themselves \textit{``Government staff has a bias that universities do not
-deliver products. However, in this project, we made many deliveries with high
-quality. Nowadays I think if we had paid the same amount for a company, it would
-not have done what we did with the quality we delivered.''}. Additionally, the
-deployment in production of each new version also \textbf{improve the
-translation of the process from one side to the other}, as mentioned by MPOG
-analyst \textit{``We had a strategic level view. When we went to the technical
-level, we had difficulty to plan each four-month release. However, in the final
-stages of the project I realized that this was not a problem because the
-deliveries were made and the results were available in production. The team was
-qualified, the code had quality and the project was well executed. So in
+big difference in the relationship because it causes empathy. You know who that
+person is, it's not simply a name.''}. This point helped to \textbf{synchronize
+the execution pace of activities}: \textit{``When we visited the lab and met
+the team, we realized that this encouraged us to validate resources faster and
+give faster feedback to the team. In return, they also quickly answered us any
+question.''}.
+
+The implementation of a Continuous Delivery pipeline also reinforced the teams'
+synchronization \cite{siqueira2018cd} .  For 81\% of interns and 75\% of senior
+developers, deploying new versions of the SPB portal in production was a
+motivator during the project. On the government side, this approach helped to
+\textbf{overcome the government bias regarding the low productivity of
+collaborative projects with academia}, as mentioned by themselves
+\textit{``Government staff has a bias that universities do not deliver
+products. However, in this project, we made many deliveries with high quality.
+Nowadays I think if we had paid the same amount for a company, it would not
+have done what we did with the quality we delivered.''}. Additionally, the
+deployment of each new version also \textbf{improve the translation of the
+process from one side to the other}, as mentioned by MPOG analyst \textit{``We
+had a strategic level view. When we went to the technical level, we had
+difficulty to plan each four-month release. However, in the final stages of the
+project, I realized that this was not a problem because the team made the
+deliveries and the results were available in production. The team was
+qualified, the code had quality, and the project was well executed. So in
 practice, our difficulty interpreting the technical details did not impact the
 releases planning.''}.
  
-\subsection{Divide the development team into priority fronts, and for each one, hire at least one specialist from the IT market}
+\subsection{Divide the development team into priority fronts, and for each one,
+hire at least one specialist from the IT market}
  
-The development team was divided into four work areas defined by the main
-demands of the project: user eXperience, devOps, integration of systems, and
-social networking. For each of them, at least one professional in the IT market
-was hired to raise the quality of the product. These senior developers were
-selected due to their experience in the open source systems and tools used in
-the project or in visual works for large scale organizations.
+The development team had four work areas divided by the main demands of the
+project: User Experience, DevOps, Integration of Systems, and Social
+Networking. For each of them, at least one professional in the IT market was
+hired to raise the quality of the product. Senior developers have a vast
+experience in the FLOSS systems and tools used in the project.
  
 The participation of senior developers in the project contributed to
-\textbf{conciliate the development processes of each institutions and make better
-technical decisions}, as quoted in one of the answers to the senior developers
-questionnaire: \textit{``I think my main contribution was to balance the
-relations between the MPOG staff and the UnB team''}. {63\%} of senior
-developers believe they have collaborated to conciliate the management and
-development process between the two institutions and also {63\%} of them that
-they helped MPOG staff to express their requests more clearly. Government
-analysts were also more open to suggestions from these developers
+\textbf{conciliate the development processes of each institution and made
+better technical decisions}, as quoted in one of the answers to the senior
+developer's questionnaire: \textit{``I think my main contribution was to
+balance the relations between the MPOG staff and the UnB team.''}. {63\%} of
+senior developers believe they have collaborated to conciliate the management
+and development process between the two institutions and also {63\%} of them
+that they helped MPOG staff to express their requests more clearly. Government
+analysts were also more open to suggestions from these developers:
 \textit{``They are developers of the upstream projects of the systems that
 integrate the platform. They conveyed trust, and then we trust in the developed
-code''}. According to questionnaire responses, they largely agreed with the
+code.''}. According to questionnaire responses, they largely agreed with the
 project development process. For 63\%, this process has close similarity to
 their previous experiences. In contrast, {62.5\%} of them did not understand
-MPOG's project management process and {50\%} believe their project productivity
-was affected by MPOG's project management process.
-
-Senior developers were also responsible for \textbf{improving the management and technical
-knowledge} of the interns about practices from industry and open source projects.
-{91\%} of the interns believe that working with professionals was important for
-learning. Working with senior developers was important during the project for all
-of them. {75\%} of senior developers believe that 'Working in pairs with a
-senior' and 63\% that 'Participate in joint review tasks' were the tasks with
-the involvement of them that most contributed to the evolution of students in
-the project. {75\%} believe that the knowledge taught by them to a intern was
-widespread among the others in the team. This acquisition of knowledge was also
-pointed by the government: \textit{``On the side of Unb, what we noticed was a
-significant improvement in the platform with the hiring of the original
-developers of the systems. They had a guide on how to best develop each feature
-and were able to solve non-trivial problems quickly.''}.
-
-Dividing the development team and hiring senior developers allowed each team
-to \textbf{self-organize and gain more autonomy in the management of their tasks}.
-Each team was coordinated by a coach who together was supported by a meta-coach
-in the execution of their activities. The coaches were points of reference
-in the development process. {89\%} of the interns said that the presence of
-the coach was essential to the sprint's running, and for {88\%} of senior
-developers coaches was essential for their interaction with the team. MPOG
-analysts saw coaches as facilitators for their activities and for communication
-with the development team. They said \textit{``I interacted
-more with the project coordinator and team coaches''}, \textit{``Usually, we
-contact a coach to clarify some requirements or to understand some feature. We
-interact more with coaches because they are more accessible than senior
-developers. Sometimes the coach would take our question to the senior
-developer''}.
+the MPOG's project management process and {50\%} believe this process could
+affect their project productivity.
+
+Senior developers were also responsible for \textbf{improving the management
+and technical knowledge} of the interns about practices from industry and open
+source projects. {91\%} of the interns believe that working with professionals
+was essential for learning. Working with senior developers was important during
+the project for all of them. {75\%} of senior developers believe that ``Working
+in pairs with a senior'' and 63\% that ``Participate in joint review tasks''
+were the tasks with the involvement of them that most contributed to the
+evolution of UnB interns in the project. {75\%} believe that the knowledge
+taught by them to a intern was widespread among the others in the team.
+Government analysts also pointed this acquisition of knowledge: \textit{``On
+the side of UnB, what we noticed was a significant improvement in the platform
+with the hiring of the original developers of the systems. They had a guide on
+how to best develop each feature and were able to solve non-trivial problems
+quickly.''}.
+
+Dividing the development team and hiring senior developers allowed each team to
+\textbf{self-organize and gain more autonomy in the management of their tasks}.
+There was a development coach to lead each team, and a ``meta-coach'' supported
+all of them in their internal management activities. The coaches (most advanced
+UnB interns) were points of reference in the development process. {89\%} of the
+interns said that the presence of the coach was essential to the sprint's
+running, and for {88\%} of senior developers coaches was essential for their
+interaction with the team. MPOG analysts saw coaches as facilitators their
+activities and communication with the development team. They said \textit{``I
+interacted more with the project coordinator (professor) and team coaches''},
+\textit{``Usually, we contact a coach to clarify some requirements or to
+understand some feature. We interact more with coaches because they are more
+accessible than senior developers. Sometimes the coach would take our question
+to the senior developer.''}.
  
 %TODO: talvez encaixar aqui a troca de papéis