Commit 7bd996bdd710d66b5f6363496f20ad811a7119f0

Authored by Melissa Wen
1 parent 1f33a8a1

[oss-2018] organize sections and improve research design

icse2018/content/01-introduction.tex
... ... @@ -51,10 +51,8 @@ government. We also validate lessons learned reported in our previous
51 51 work \cite{meirelles2017spb}.
52 52  
53 53 % TODO: Verificar as seções
54   -Section \ref{sec:relatedwork} describes related work. Section
55   -\ref{sec:casestudy} gives a brief description of the case study. Section \ref{sec:researchdesign}
56   -describes our research questions and methodology. Section \ref{sec:discussion}
57   -presents findings derived from our quantitative and qualitative analyses.
58   -Section \ref{sec:results} we describe the results. Finally, we present the
59   -limitations, related work and conclusions.
60   -
  54 +Section \ref{sec:relatedwork} describes related work. Section
  55 +\ref{sec:researchdesign} describes our research questions and research
  56 +methodology with a brief description of the case study. Section \ref{sec:results}
  57 +presents results derived from our quantitative and qualitative analyses.
  58 +Finally, we discuss our findings and future work in section \ref{sec:discussion}.
... ...
icse2018/content/04-case.tex
... ... @@ -1,45 +0,0 @@
1   -\section{The Case Study}
2   -\label{sec:case}
3   -
4   -The project to evolve the Brazilian Public Software Portal
5   -\cite{meirelles2017spb} was a partnership between government and academia held
6   -between 2014 and 2016. To solve maintenance problems and fill
7   -design-reality gaps in the portal, the Ministry of Planning (MPOG) joined the
8   -University of Brasília (UnB) and the University of São Paulo (USP) to develop a
9   -platform with features and technologies novelties in the government context.
10   -
11   -%TODO: - Ainda não se falou de ferramentas integradas. Deve ser apresentado o novo SPB (de forma simples e direta como fizemos no IEEE) para se entender a complexidade do projeto/caso
12   -
13   -The academic team carried out development activities in the Advanced Laboratory
14   -of Production, Research and Innovation in Software Engineering (LAPPIS) of UnB. The
15   -project management and development process in this laboratory is usually
16   -executed adopting free software practices and agile approach. For this project, a total of 42 undergraduate students, two MSc
17   -students and two coordinator-professors participated in the development team.
18   -Six IT professionals were also hired as senior developers due their vast
19   -experiences in Front-end/UX or in one of the softwares integrated to the
20   -platform.
21   -
22   -The government team was composed of a director, a coordinator, and two IT
23   -analysts from a department of MPOG. Although it was responsible for the
24   -execution of this collaboration project, this department generally does not
25   -execute development of ministry's software. This department is responsible for
26   -contracting and homologating software development services and follows
27   -traditional management approaches, such as the RUP.
28   -
29   -These two aforementioned teams
30   -periodically met in person for the purpose of managing the project progress. These meetings initially only took place at the
31   -ministry's headquarters to discuss strategic/political and technical goals.
32   -These meetings were held monthly with the presence of two UnB professors, the
33   -executive-secretary of the Presidency (project supporter) and all MPOG members
34   -responsible for the project. The management of the development team was
35   -concentrated in the academia side. The workflow was organized in Redmine in
36   -biweekly sprints and 4-month releases, with intermediate deliveries hosted in
37   -university environment. However, with the progress of the project, this format
38   -proved to be inefficient. Conflicts between the internal management processes
39   -and differences in pace and goals of each institution were compromising the
40   -platform development.
41   -
42   -In this case study, we focus on analyzing the dynamics between government and
43   -academia for collaborative development. We aim to map the practices adopted in
44   -the project management and development process to harmonize the cultural and
45   -organizational differences of the institutions involved.
46 0 \ No newline at end of file
icse2018/content/04-methods.tex 0 → 100644
... ... @@ -0,0 +1,118 @@
  1 +\section{Research Design}
  2 +\label{sec:researchdesign}
  3 +
  4 +Our analysis was guided by the following research questions:
  5 +
  6 +\textbf{RQ1.}{What practices based on open source development experiences would
  7 +help to combine teams with different management processes in a
  8 +government-academia collaboration project?}
  9 +
  10 +
  11 +\textbf{RQ2.}{How do open source development practices benefit the process of
  12 +developing an e-government platform in a government-academia collaboration?}
  13 +
  14 +To answer these questions, we use as a case study the evolution project of the
  15 +SPB portal \cite{meirelles2017spb}, a government and academia collaborative
  16 +development based on open source software integration. From this project, we
  17 +collect public data from the project development environment available on the
  18 +developed platform itself, and conduct two surveys and an interview aimed at the
  19 +different roles performed by the ex-project participants, as detailed in the
  20 +following subsections
  21 +
  22 +\subsection{The case estudy}
  23 +
  24 +The project to evolve the Brazilian Public Software Portal
  25 +\cite{meirelles2017spb} was a partnership between government and academia held
  26 +between 2014 and 2016. To solve maintenance problems and fill
  27 +design-reality gaps in the portal, the Ministry of Planning (MPOG) joined the
  28 +University of Brasília (UnB) and the University of São Paulo (USP) to develop a
  29 +platform based on the integration and evolution of existing open-source
  30 +software, with many features and technologies novelties in the government
  31 +context.
  32 +
  33 +The academic team carried out development activities in the Advanced Laboratory
  34 +of Production, Research and Innovation in Software Engineering (LAPPIS) of UnB. The
  35 +project management and development process in this laboratory is usually
  36 +executed adopting free software practices and agile approach. For this project, a total of 42 undergraduate students, two MSc
  37 +students and two coordinator-professors participated in the development team.
  38 +Six IT professionals were also hired as senior developers due their vast
  39 +experiences in Front-end/UX or in one of the softwares integrated to the
  40 +platform.
  41 +
  42 +The government team was composed of a director, a coordinator, and two IT
  43 +analysts from a department of MPOG. Although it was responsible for the
  44 +execution of this collaboration project, this department generally does not
  45 +execute development of ministry's software. This department is responsible for
  46 +contracting and homologating software development services and follows
  47 +traditional management approaches, such as the RUP.
  48 +
  49 +These two aforementioned teams
  50 +periodically met in person for the purpose of managing the project progress. These meetings initially only took place at the
  51 +ministry's headquarters to discuss strategic/political and technical goals.
  52 +These meetings were held monthly with the presence of two UnB professors, the
  53 +executive-secretary of the Presidency (project supporter) and all MPOG members
  54 +responsible for the project. The management of the development team was
  55 +concentrated in the academia side. The workflow was organized in Redmine in
  56 +biweekly sprints and 4-month releases, with intermediate deliveries hosted in
  57 +university environment. However, with the progress of the project, this format
  58 +proved to be inefficient. Conflicts between the internal management processes
  59 +and differences in pace and goals of each institution were compromising the
  60 +platform development.
  61 +
  62 +\subsection{Survey}
  63 +
  64 +We divided the UnB development team into two groups of respondents according to
  65 +their roles during the project: UnB Interns and Senior Developers. For each
  66 +group, we designed an online survey with topics related to project organization,
  67 +development process, communication and relationship between members, acquired
  68 +knowledge and experience with free software.
  69 +
  70 +\begin{enumerate}
  71 + \item \textit{UnB interns:} 42 undergraduate students who
  72 +participated in any time of the project as developer and received scholarship.
  73 +We received a total of 37 responses. Their average age is 25 years old and
  74 +91.9\% of them are male. Currently, 35.1\% continue at university as
  75 +undergraduate or graduate students, 18.9\% work as developer in a small company
  76 +and 18.9\% in medium or large companies, 10.8\% are entrepreneurs, 8.1\% are
  77 +unemployed and the others work as teachers or civil servants. 43.2\% said the
  78 +SPB project was their first experience with free software.
  79 +
  80 + \item \textit{Senior Developers:} eight advanced level researchers, MSc
  81 +students or IT market professionals who participated in some period of the
  82 +project. All of them answered the questionnaire. Their average age is 32 years
  83 +old and 87.5\% are male. They have an average of 11 years of experience in the
  84 +IT market, and currently 62.5\% of respondents are company employees, 37.5\% are
  85 +freelance developers, 25\% are master's degree students and 25\% entrepreneurs.
  86 +They have worked on average in 5 companies and participated in 4 to 80 projects.
  87 +They participated in this collaborative project between 7 to 24 months. 85.7\%
  88 +of them had some experience with free software before the SPB project.
  89 +\end{enumerate}
  90 +
  91 +\subsection{Interview}
  92 +
  93 +On the government side, two MPOG IT analysts were interviewed separately. They
  94 +were selected because they were the only government representatives who
  95 +interacted directly with the development team and project management process.
  96 +Each interview took an average of 2 hours with 28 open questions classified into
  97 +fours parts: Professional profile; Organization, communication and development
  98 +methodologies in the context of government and project; Satisfaction with the
  99 +developed platform; Lessons learned. They are more than 30 years old and have
  100 +been government employees for more than 7 years. Only one of them continues
  101 +working in the same ministry. For both, this collaborative project was their
  102 +first experience of government-academia development collaboration.
  103 +
  104 +\subsection{Data Collection}
  105 +
  106 +We quantitatively analyze data about the development of the project using data
  107 +publicly available on the SPB platform. We collect from the repository manager
  108 +of the platform, Gitlab - integrated platform software tool, all open issues
  109 +and commits made between April 2015 to February 2016 and related to the
  110 +main repository of the platform, that is, the development repositories of the
  111 +integrated software were not considered. For issues, the data collected were:
  112 +project name, author of the issue, opening date, issue title and number of
  113 +comments. We also collected informations about: total open issues, total
  114 +commits, different authors of issues, total of different authors of issues,
  115 +total of comments, authors of comments, total of authors other than comments.
  116 +
  117 +
  118 +% And finally, we analized Colab code before and after the project to evaluate how much effort was spent to use this software as a component of the platform.
... ...
icse2018/content/05-methods.tex
... ... @@ -1,101 +0,0 @@
1   -\section{Research Design}
2   -\label{sec:researchdesign}
3   -
4   -Our analysis was guided
5   -by the following research questions:
6   -
7   -\textbf{RQ1.} {How to well combine teams with different management processes
8   -in a government-academia collaboration project?}
9   -
10   -In this first moment, we describe what changes in the management model and the
11   -development process have improved interactions between institutions, as well as
12   -internally. To map the benefits obtained by these movements, we use evidence
13   -obtained from interviews and online surveys with members on both sides, after
14   -project closure. We also collect data from management and communication tools
15   -used throughout the project.
16   -
17   -In a second moment, we address our analysis to issues related to organizational
18   -differences and diversity of project members in terms of maturity and experience
19   -in collaborative development. The harmony between teams sought not only to
20   -approximate the mind-set and culture of teams but also to delimitate the
21   -interactions between different roles and responsibilities. Evaluating this
22   -synergy generates the second research question:
23   -
24   -\textbf{RQ2.} \textit{Which boundaries should be established between government
25   -and academia teams in collaboration interactions?}
26   -
27   -We highlight positive and negative effects of boundaries created among project
28   -member using evidences from interview responses and open field responses from
29   -online surveys.
30   -
31   -To answer the two research questions presented, we
32   -designed an interview and two questionnaires with quantitative and
33   -qualitative questions addressed to project members. We also collect data from
34   -tools that supported the project management activities.
35   -
36   -\subsection{Surveys}
37   -
38   -We conducted after-project surveys divided into three target groups of
39   -project participants:
40   -
41   -\begin{enumerate}
42   - \item \textit{MPOG Staff:} two government-side employees who have acted
43   -directly in the platform development process. They were separately interviewed and each interview took an average
44   -of 2 hours with 28 open questions divided by subject: Professional profile;
45   -Organization, communication and development methodologies in the context of
46   -government and project; Satisfaction with the developed platform; Lessons
47   -learned.
48   - \item \textit{UnB undegraduated students:} 42 undergraduate students who
49   -participated in any time of the project as developer and received scholarship. A
50   -questionnaire with 45 closed and six open questions was sent through emails using
51   -online form platform. The topics covered were: Organization, communication
52   -and development activities between the respondents and the different groups of
53   -the project; Learning acquired; Professional learning; Experience with free software
54   -projects. We received a total of 37 responses.
55   - \item \textit{Senior Developers:} eight advanced level researchers, MSc students or
56   -IT market professionals who participated in some period of the project. A
57   -questionnaire with 29 closed questions and 10 open questions addressed the
58   -follow topics: Organization, communication and relationship between respondents
59   -and distinct groups of the project; Development process; Experience with Free
60   -Software. All eight recipients answered the questions.
61   -\end{enumerate}
62   -
63   -\subsection{Data Collection}
64   -
65   -%TODO: quais dados?
66   -In a second round, we also collected post-mortem data from Gitlab - an open source and web-based repository manager integrated to SPB platform
67   -used for management, communication and code versioning during the 30-month
68   -project. These all data are open and available
69   -for access at any time on the SPB Portal. This data analyze composes and ratifies
70   -the evidences obtained in the previous round (surveys). The results
71   -represent, in terms of volume, interactions and the evolution of these
72   -interactions between the government and academia teams, and, in terms of
73   -development complexity, the platform size and quantity of software releases
74   -delivered.
75   -
76   -\subsection{Respondents profile}
77   -
78   -\subsubsection{MPOG Staff}
79   -
80   -The two analysts interviewed are more than 30 years old and have been government
81   -employees for more than 7 years. Only one of them continues working in the same
82   -ministry. Both reported that the collaborative project studied was their first
83   -experience in collaborative projects between government and academia.
84   -
85   -\subsubsection{UnB undergraduated students}
86   -
87   -The average age of the 37 respondents is 25 years old and 91.9\% of them are male.
88   -Currently, 35.1\% continue at university as undergraduate or graduate students,
89   -18.9\% work as developer in a small company and 18.9\% in medium or large
90   -companies, 10.8\% are entrepreneurs, 8.1\% are unemployed and the others work as
91   -teachers or civil servants.
92   -
93   -\subsubsection{Senior Developers}
94   -The average age is 32 years old and 87.5\% are male. They have an average of 11
95   -years of experience in the IT market, and currently 62.5\% of respondents are
96   -company employees, 37.5\% are freelance developers, 25\% are master's degree
97   -students and 25\% entrepreneurs. They have worked on average in 5 companies and
98   -participated in 4 to 80 projects. They participated in the collaborative project
99   -studied between 7 to 24 months.
100   -
101   -% And finally, we analized Colab code before and after the project to evaluate how much effort was spent to use this software as a component of the platform.
icse2018/content/05-results.tex 0 → 100644
... ... @@ -0,0 +1,222 @@
  1 +\section{Results}
  2 +\label{sec:results}
  3 +
  4 +%TODO: Talvez esse paragráfo tem que está no Research Design
  5 +%%
  6 +The case study was analyzed and divided into two phases according to the project
  7 +management model. In the second phase (after one year of execution), several
  8 +practices have been applied to harmonize the cultural and organizational
  9 +divergences of the institutions involved.
  10 +%%
  11 +At the end of the project, an empirical
  12 +model of management and development process was created by aligning experiences
  13 +from the FLOSS universe, academic research and bureaucracies needed by the
  14 +government. In this section, we present by context the practices adopted in this
  15 +second phase and show the benefits generated by its deployment, as summarized
  16 +in the Table \ref{practices-table}.
  17 +
  18 +%% TODO: explicar a estrutura e cada campo da tabela
  19 +
  20 +\begin{table}[]
  21 +\centering
  22 +\resizebox{\textwidth}{!}{%
  23 +\begin{tabular}{ | m{4cm} m{10cm} m{10cm} | }
  24 +\hline
  25 +\textbf{Decision} & \textbf{Practice Explanation} & \textbf{Benefits} \\ \hline
  26 +\textbf{Project management and communication on the developing platform itself}
  27 +&
  28 +\begin{itemize}
  29 +\item Migration of project management and communication into
  30 +the platform under development using its integrated software components Gitlab
  31 +and Mailman
  32 +\end{itemize} &
  33 +\begin{itemize}
  34 +\item Trusting developed code;
  35 +\item Communicating with transparency and efficiency;
  36 +\item Easier monitoring and increasing interactions between development team and public servants;
  37 +\item Producting organically documentation and records;
  38 +\end{itemize} \\ \hline
  39 +
  40 +\textbf{Bring together government staff and development team} &
  41 +\begin{itemize}
  42 +\item Biweekly gov staff, senior developers and coaches met to planning and
  43 +review sprint at the UnB headquarters. \item Most of features under development
  44 +were discussed on Gitlab Issue Tracker. \item Only strategic decisions or
  45 +bureaucratic issues involve board directors. \item Continuous Delivery
  46 +\end{itemize} &
  47 +\begin{itemize}
  48 +\item Reducing communication misunderstood and better meeting expectations of both sides;
  49 +\item Overcoming the government bias regarding low productivity of collaboration with academia;
  50 +\item Aligning both activities execution pace;
  51 +\item Improving the translation of the process from one side to the other;
  52 +\end{itemize} \\ \hline
  53 +
  54 +\textbf{Split development team into priority work fronts with IT market specialists} &
  55 +\begin{itemize}
  56 +\item The development was divided into four fronts (DevOps / UX / Noosfero / Colab) with a certain self-organization of tasks.
  57 +\item IT market professionals with recognized experience on each front were hired to work in person or remotely.
  58 +\item For each front, there was at least one senior developer and the role of coach.
  59 +\item The meta-coach role was also defined to coordinate tasks between teams.
  60 +\end{itemize} &
  61 +\begin{itemize}
  62 +\item Helping to reconciliate development processes and decision-making;
  63 +\item Creating support-points for students, senior developers, and gov staff;
  64 +\item Transfering of knowledge from industry and FLOSS community to both academia and government;
  65 +\end{itemize}\\ \hline
  66 +\end{tabular}%
  67 +}
  68 +\caption{Empirical SPB management method and its benefits}
  69 +\label{practices-table}
  70 +\end{table}
  71 +
  72 +\subsection{Project management and communication on the developing platform
  73 +itself}
  74 +
  75 +After nine months of project activities, the first version of new SPB Portal was
  76 +released. From this point, we started to migrate the management and
  77 +communication interactions to the platform under development. In short, Wiki
  78 +feature was used for meeting logging, defining goals, sprint planning, and
  79 +documentation of deployment processes and administration resources guide. Issue
  80 +tracker was used for discussing requirements, monitoring the features under
  81 +development, registering changes, and validating functionalities delivered. Finally, the
  82 +whole team used Mailing list to defining schedules of meetings and deliveries
  83 +and also to collaborative definition of requirements.
  84 +
  85 +Our surveys reports Mailing list (100\%) and Issue Tracker (62.5\%) as the main means
  86 +of interaction between senior developers and undergraduates. Developers and MPOG
  87 +staff also interacted mostly via Mailing List (87.5\%) and Issue tracker (50\%).
  88 +According to research findings, this movement made \textbf{communication more
  89 +transparent and efficient}. A MPOG IT analyst said that the
  90 +\textit{``Communicating well goes far beyond the speed, it is someone being able
  91 +to communicate to everyone everything that is happening in the project. We did
  92 +not use emails. We use more mailing list and avoid e-mails. It helped a lot
  93 +because everything was public and did not pollute our mailbox. You wanted to
  94 +know something, could go there and look at what was happening''}.
  95 +
  96 +Migrating to SPB platform also provided an \textbf{easier monitoring and
  97 +increase interactions between development team and public servants by
  98 +coordinators}. As shown by collected data, in the last 15 months of the project,
  99 +775 issues and 4,658 issue comments was registered in the main repository
  100 +(without considering the software repositories that integrated the platform)
  101 +within the SPB platform. The issues have 59 different authors (8 from MPOG
  102 +staff), and commented by 64 different users (9 form MPOG staff and users).
  103 +Considering issues with higher level of interaction those that have 10 or more
  104 +comments, in a set of 84 issues, MPOG staff authored 36 issues (which represents
  105 +about 43\% of these most active issues). A MPOG analyst highlighted that
  106 +\textit{``there was a lot of evolution, a lot of communication via Gitlab''}.
  107 +This interaction also led MPOG staff to \textbf{trust developed code}:
  108 +\textit{``Everything was validated, we tested the features and the project was
  109 +developed inside the platform, so that the feature was validated in the
  110 +development of the software itself. From the moment we installed it, and
  111 +began to use it for development, this validation was constant. We felt confident
  112 +in the features''}.
  113 +
  114 +One of the main concerns of traditional approach is meticulous documentation of
  115 +the software designed and the development steps. With this aforementioned
  116 +decision, we could meet this government demand without bureaucracies and changes
  117 +in our development process, \textbf{producting organically documentation and
  118 +records} in the platform itself, as one of the MPOG response evidenced:
  119 +\textit{``For me, it was a lot of learning. There is a lot of things documented
  120 +in the e-mails and also in the portal itself. At any moment we can go there and
  121 +see how it worked, how someone did something. We can recover those good points''}.
  122 +
  123 +\subsection{Bringing together government staff and development team}
  124 +
  125 +The MPOG analysts observed communication noise in the dialogue between them and
  126 +their superiors and in the dialogues with the development team that were
  127 +intermediated by the superiors. They said that direct dialogue with the
  128 +development team and biweekly visits to the university's lab \textbf{reduce
  129 +communication misunderstood}: \textit{``At this point, the communication started to
  130 +change.. started to improve''}. According to another interviewee, this new
  131 +dynamic unified the two sides: \textit{``I believe it was very positive, we also liked to
  132 +go there, to interact with the team. I think it brought more unity, more
  133 +integration into the project''}. The participation of the MPOG staff was also
  134 +considered positive by {72.9\%} of the undegraduates and to {81.1\%} of them
  135 +think the presence of MPOG staff in sprint ceremonies was important for the
  136 +development. In addition, to \textbf{better meet expectations of both sides}
  137 +regarding the requirements developed, {75.6\%} of students believe that writing
  138 +the requirements together with the MPOG staff was very important. According to
  139 +one of them \textit{``Joint planning and timely meetings were very important for
  140 +understanding the needs of MPOG''}.
  141 +
  142 +An imported consequence of this direct government-academia interaction in
  143 +laboratory was empathy, as reported by one of the interviewees \textit{``You
  144 +know people in person and it makes such a big difference because it causes
  145 +empathy. You already know who that person is, it's not just a name''}. This
  146 +subjectively helped to \textbf{align both activities execution pace},
  147 +\textit{``When we went there, we knew the people and we realized that, on our
  148 +side, we also felt more encouraged to validate faster and give faster feedback
  149 +to the teams [..] We gave this feedback fast and they also gave quick feedback
  150 +for any our questions''}. The teams' synchronization was reinforced with the
  151 +implementation of a Continuous Delivery pipeline. The benefits of this approach
  152 +were presented in our previous work \cite{siqueira2018cd} and corroborate these research
  153 +results. To 81.1\% of students and 75\% of senior developers, deploying new
  154 +versions of the SPB portal in production was a motivator during the project.
  155 +
  156 +One of the MPOG analyst interviewed also noted these releases also helped to
  157 +\textbf{overcome the government bias regarding low productivity of collaborative
  158 +projects with academia}: \textit{``At first, the government staff had a bias that
  159 +universities do not deliver. We overcame that bias in the course of the project.
  160 +We deliver a lot and with quality. Today, I think if we had paid the same amount
  161 +for a company, it would not have done what was delivered and with the quality
  162 +that was delivered with the price that was paid''}. Additionally, the deployment
  163 +in production of each new version also \textbf{improve the translation of the
  164 +process from one side to the other}, as mentioned by MPOG analyst \textit{``We had an
  165 +overview at the strategic level. When we went down to the technical level, plan
  166 +the release every four months was difficult. But in the end, I think this has
  167 +not been a problem. A project you are delivering, the results are going to
  168 +production, the code is quality, the team is qualified/capable and the project
  169 +is doing well, it does not impact as much in practice''}.
  170 +
  171 +\subsection{Split development team into priority work fronts with IT market
  172 +specialists}
  173 +
  174 +Four teams were formed to dedicated to the main development demands of the
  175 +portal: UX, DevOps, System-of-Systems, and Social Networking. External
  176 +developers with vast experience in the SPB platform software components and
  177 +professionals with experience in front-end and UX were hired. These
  178 +professionals also contributed to disseminate practices adopted in the industry
  179 +and in the free software communities to other project members. {87.5\%} of
  180 +seniors agreed with our project development process. For 62.5\% this process
  181 +has a good similarity to their previous experiences. Their experience
  182 +\textbf{helped to reconcile development processes and decision making}, as
  183 +stated by one of the respondent developers \textit{``I think my main
  184 +contribution was to have balanced the relations between the MPOG staff and the
  185 +UnB team''}. {62.5\%} of senior developers believe they have collaborated in the
  186 +relationship between the management and development processes of the two
  187 +institutions and {62.5\%} asserted that helped MPOG staff to more clearly
  188 +express their requests. {62.5\%} of them did not understand MPOG's project
  189 +management process and {50\%} believe their project productivity was affected
  190 +by MPOG's project management process. For the government, these professionals
  191 +gave credibility to the development \textit{``You had the reviewers, who were
  192 +the original developers of the software, that gave you confidence and
  193 +confidence in the code''}.
  194 +
  195 +In addition, with these professionals was possible to \textbf{transferred
  196 +knowledge of industry and free software to government and academia}. Working
  197 +with senior developers was important for all interns during the
  198 +project. {91\%} of them also believe that working with professionals was
  199 +important for learning. {75\%} of senior developers believe that 'Working in
  200 +pairs with a senior' and 62.5\% that 'Participate in joint review tasks' were
  201 +the tasks with the involvement of them that most contributed to the evolution
  202 +of students in the project. And, in guiding a students, {75\%} believe that
  203 +this knowledge was widespread among the others in the team. This acquisition
  204 +of knowledge was also noted by the government, which stated \textit{``On the side of
  205 +UnB, what we perceived was that the project was very big leap when the
  206 +original software developers were hired in the case of Noosfero and Colab,
  207 +because they had a guide on how to develop things in the best way and were
  208 +able to solve non-trivial problems and quickly''}.
  209 +
  210 +The fronts also gained more autonomy to manage their activities. The role
  211 +of ``meta-coach'' was defined among the students, to coordinate the interactions
  212 +between teams and coach to coordinate each front. Coaches have become a \textbf{point
  213 +of reference for the development process}. {89.1\%} of students said that the
  214 +presence of the coach was essential to the running of sprint, and for {87.5\%}
  215 +of senior developers coaches was essential for their interaction with the team.
  216 +MPOG analysts saw coaches as facilitators for their activities and for
  217 +communication with the development team. One of the interviewees said \textit{``I
  218 +interacted more with the project coordinator and team coaches''}, \textit{``The reason
  219 +for this was that the coaches were more likely to meet the requirements, to
  220 +ask questions about requirements, to understand some features. interaction with
  221 +leaders than with senior developers. Sometimes the coaches brought the question
  222 +to the senior developers''}.
... ...
icse2018/content/06-discussion.tex 0 → 100644
... ... @@ -0,0 +1,113 @@
  1 +\section{Discussion and Final Remarks}
  2 +\label{sec:discussion}
  3 +
  4 +In this paper we examined the empirical model built in a collaborative project
  5 +between government and academia that successfully harmonized the differences in
  6 +the common approaches to software development management used by each side. We
  7 +mapped the key decisions made over the 30-months of the project, that aimed to
  8 +improve communication and the development process as a whole. We also elaborated
  9 +two surveys and one interviews that were conducted separately for three groups
  10 +of participants. We obtained a total of responses of 37 undergraduated
  11 +students, eight IT market professionals, and two government officials. Finally,
  12 +we collected post-mortem public data on project management carried out on the
  13 +platform itself. The results revealed nine practices were developed from three
  14 +main decisions taken and 11 benefits were obtained with the adoption of these
  15 +practices.
  16 +
  17 +In our previous work \cite {meirelles2017spb}, we presented the unprecedent
  18 +platform developed in the case study project and seven lessons learned taking into account only the
  19 +academia-side view. The new results acquired in the current work corroborate
  20 +with these lessons, adding the point of view of the government and the academia
  21 +in diverse performed levels. In addition, these results suggest that many free
  22 +software development practices can be replicated in other contexts in which the
  23 +diversity and plurality of its stakeholders need to be leveled and reconciled.
  24 +
  25 +The results obtained also showed questions that were not overcome during the
  26 +project and which we believe need to be evaluated for future collaborations
  27 +between government and academia for software development:
  28 +\begin {itemize}
  29 +\item Improving understanding about collaboration: \textit{"During development,
  30 +we realized that the development team also felt like the owner of the project,
  31 +not just a mere executor. partnership, then it had a lot of that team issue to
  32 +suggest things to be put into the project. It was not a customer relationship it
  33 +was a partnership relationship, so there was a lot of issue suggesting by the
  34 +team to be put into the project"}
  35 +\item Discussion of roles and responsibilities: \textit{"Who had the power to
  36 +make a decision? There was no one, because it was a very equal relationship. The
  37 +two organs were on the same hierarchical level within the work plane. But this
  38 +does not work well, you have to leave well defined to whom the last word belongs
  39 +in the decisions, because the conflicts will always happen."}.
  40 +\item Look for a balance in the requirements definition. The responses showed
  41 +that the government felt that it was not detailed enough and the development
  42 +team felt that the requirements needed to be matured with the use.
  43 +\item Smoothing the intermediations between the different roles \textit{"When we
  44 +had the [UnB] coordinator, when we forwarded a direct question to a developer,
  45 +the coordinator responded. So that was negative, because we felt a little
  46 +coerced from talking directly to the teams"}
  47 +\end {itemize}
  48 +
  49 +As future work, we will reapply in another government-academia paternship
  50 +project the practices evidenced in this case study, and conduct
  51 +qualitative and quantitative research throughout its execution. We intend to
  52 +prove the effectiveness in adopting free software development practices to
  53 +align the demands and expectations of a G-A collaboration.
  54 +
  55 +\begin{comment}
  56 +
  57 +\begin{itemize}
  58 +\setlength\itemsep{1em}
  59 +\item \textit{How to involve students real-world projects, interacting with real customers}
  60 +Em uma das perguntas abertas na pesquisa direcionada aos alunos, os respondentes abordaram a importância que o projeto teve na sua vida profissional: "O projeto me permitiu compreender um pouco sobre a dinâmica do mercado de software em Brasília e, em especial, dentro do serviço público.", "Ótima experiência. Agregou muito valor à minha formação acadêmica e profissional. Replico e repasso os conhecimentos adquiridos ainda hoje, mesmo com mais de um ano do término do projeto.", "Uma das características mais interessantes do projeto foi proporcionar aos alunos uma experiência de desenvolvimento de software muito similar ao que eles escontram em empresas. Isso inclui ter contato com clientes reais, planejar o ciclo de desenvolvimento e participação em times."; "Apesar dos cursos de graduação abordar teorias sobre desenvolvimento de software, a aplicação em um projeto real teve muito impacto positivo no meu desenvolvimento como profissional e me forneceu uma visão de mercado."
  61 +
  62 +\item \textit{The participation of experienced professionals is crucial to succcess of the project}
  63 +Além dos resultados positivos sobre a importância da presença dos desenvolvedores seniors na visão dos alunos, em resposta aberta seus benefícios foram ratificados: ""Esse projeto foi essencial na minha formação. O contato prático com o mundo real e pessoas experientes, alinhado a metodologias flexíveis auto organizadas para o desenvolvimento de software proporcionou a produção de resultados rápidos e uma equipe bastante motivada, algo essencial para o sucesso de um projeto.", "Além de me permitir aprender com profissionais mais experientes e em diferentes áreas do processo de desenvolvimento de software."
  64 +Este tópico também foi relatado do lado do governo: "Você tinha os revisores, que eram os desenvolvedores originais dos softwares, isso passava confiança e a gente sentia confiança no código."
  65 +Por fim, os desenvolvedores senior relataram uma boa relação de aprendizagem com os alunos: "A relação com os alunos era respeitosa. Os alunos estavam abertos e parecia que se sentiam confortáveis para receber e dar orientações e sugestões."
  66 +
  67 +\item \textit{A balanced relationship between academia and industry}
  68 +Um dos desenvolvedores trouxe a tona a prioridade dos compromissos escolares em relação ao projeto: "Dificuldades na comunicação presencial da equipe, por conta de diferentes horários dos alunos. Ausência de alunos em momentos importantes do projeto, por conta de provas e outras demandas universitárias."
  69 +Alguns alunos abordaram o avanço profissional e técnico durante o projeto: "A equipe trabalhou em quase todos os momentos no que seria o estado da arte da Engenharia de Software", "O projeto como um todo trouxe bastante experiência técnica para quem estava iniciando no mundo da engenharia de software. Eu e outros colegas atingimos níveis de conhecimento que nos possibilitaram engajar em projetos de software de maneira muito mais impactante e confiante, participando das comunidades dos softwares utilizados e até nos tornando membros mais influentes, com permissão de realizar contribuições diretas no código oficial das plataformas, o que é algo que inspira e motiva ainda mais a colaboração entre membros da equipe e das comunidades de software livre envolvidas."
  70 +
  71 +\item \textit{Managing different organizational cultures}
  72 +Os entrevistados do lado do governo retratam o problema de tradução entre os modelos adotados pelo lado do governo e pelo lado da academia: "Não era uma tarefa muito fácil, fazer este DE-PARA, porque a ferramenta de acompanhamento de projeto lá do MPOG é amarrada no modelo tradicional de gerenciamento de projetos", " A gente tinha que documentar, pedir um report para vocês [UnB]. Faziamos um processo de atualização meio que em conjunto, para chegar no equilíbrio pra levar pra reunião do projeto."
  73 +
  74 +\item \textit{Manage higher-level and lower-level goals separately}
  75 +Não encontrei
  76 +
  77 +\item \textit{Living will ill-advised political decisions}
  78 +Os funcionários do MPOG avaliam os impactos negativos de decisões técnicas tomadas com viés político: "Realmente, a questão do Colab foi o mais impactante [na complexidade] deu um impacto muito grande. Na prática, tomou essa decisão e gastou-se meses para poder fazer ele conversar com as outras ferramentas e esse tempo poderia ter sido gasto fazendo módulos novos, por exemplo, o que a gente tinha previsto, o Mercado Público, que é a questão dos prestadores de serviço lá dentro da ferramenta. Não conseguimos fazer. A gente teve que remover isso do escopo porque não teve não sobrou tempo. A gente gastou tanto esforço evoluindo o Colab (fazendo API, fazendo integração) que não sobrou tempo para fazer. Um negócio que tinham no portal antigo e no novo acabou que não foi feito.", Então ficar atento as premissas, detalhar melhor
  79 +os requisitos e tentar o mínimo de influencia política num projeto. Porque essas influencias politicas, elas atrapalham demais, elas criam soluções insustentáveis. Esse caso aí do Colab não é simples. Colab, virou um exemplo, um exemplo de influências políticas que na hora pode ter feito sentido, mas depois gera muito overhead, dor de cabeça."
  80 +
  81 +\item \textit{Consider sustainability from the beginning}
  82 +Durante a entrevista, um dos entrevistados do governo ressaltou a preocupação com sustentabilidade: "Porque você pode fazer o negócio mais perfeito possível, mas se você não manter là frente, você tem um problema. Se você não consegue manter, ali na frente você não vai ter uma equipe tão boa para manter, você vai ter um problema."
  83 +\end{itemize}
  84 +
  85 +In addition to ratifying the lessons learned, some of the questionnaires or the
  86 +interview answers raise negative points that need to be evaluated and better
  87 +worked for future collaboration between government and academia. These negative
  88 +responses are mainly related to differences in the management approaches of each
  89 +organization.
  90 +
  91 +Do lado do governo:
  92 +
  93 +\begin{itemize}
  94 +\item \textbf{Detalhamento dos requisitos}
  95 +\subitem "Eu acho que é este detalhamento de requisito, eu acho que a gente tinha que exigir mais e melhor detalhamento dos requisitos."
  96 +\subitem "Eu acho que, no final, a gente já tava numa liga legal no sentido de ter os requisitos num nível de detalhes necessários. Eu sempre achava que os requisitos não estavam documentados o suficiente pra a visão do cliente na hora do desenvolvimento. A gente desenvolvia, mas, às vezes, não era bem exatamente aquilo que o cliente queria."
  97 +\subitem "Nem sempre quando o pedido de mudança partia da equipe isso era debatido ou acordado. Às vezes a gente só descobria lá no final da sprint. A gente tinha alguns probleminhas com isso. [..] Você precisa, sei lá, fazer uma entrevista, detalhar isso melhor, porque você vai codificar e você precisa saber. Ás vezes isso não acontecia e aí o programador fazia da forma que ele achava. Na hora da entrega, na hora de colocar no ar, a gente tinha algumas surpresas, algumas coisas que não estavam legais."
  98 +\item \textbf{Discussão de papéis e responsabilidades}
  99 +\subitem "Quem que tinha o poder de tomar uma decisão? Não tinha, porque era uma relação muito igual. Os dois órgãos ficavam no mesmo nível hierárquico dentro do plano de trabalho. Mas isso não funciona bem, você tem que deixar bem definido a quem pertence a última palavra nas decisões, porque os conflitos sempre vão acontecer."
  100 +\item \textbf{Estabelecer limites sem coagir}
  101 +\subitem "Quando tinha o coordenador (que coordenava as equipes), o que sentiamos de ruim era que, quando a gente encaminhava uma pergunta direto para um desenvolvedor, o coordenador respondia. Então isso era meio negativo, porque causava uma dificuldade, a gente se sentia meio coagido de conversar direto com as equipes.
  102 +\end{itemize}
  103 +
  104 +Do lado da academia:
  105 +
  106 +\begin{itemize}
  107 +\item bla
  108 +\end{itemize}
  109 +
  110 +% "No final teve aquele medo, aquela pressão, que não saia recurso. Teve uma desmobilização grande no final porque não estava saindo recurso, teve que liberar os seniors, isso também atrapalhou. Então essa parte aí de política impacta bastante no nosso trabalho no que era entregue. Naquela época, as vezes até tinha o recurso, mas tinha uma resistência da alta administração, agora não tem o recurso."
  111 +
  112 +\end{comment}
  113 +
... ...
icse2018/content/06-results.tex
... ... @@ -17,10 +17,9 @@ in the Table \ref{practices-table}.
17 17  
18 18 %% TODO: explicar a estrutura e cada campo da tabela
19 19  
20   -\begin{table}[]
21   -\centering
  20 +\begin{table}[h!]
22 21 \resizebox{\textwidth}{!}{%
23   -\begin{tabular}{ | m{4cm} m{10cm} m{10cm} | }
  22 +\begin{tabular}{cc}
24 23 \hline
25 24 \textbf{Decision} & \textbf{Practice Explanation} & \textbf{Benefits} \\ \hline
26 25 \textbf{Project management and communication on the developing platform itself}
... ... @@ -63,10 +62,10 @@ bureaucratic issues involve board directors. \item Continuous Delivery
63 62 \item Creating support-points for students, senior developers, and gov staff;
64 63 \item Transfering of knowledge from industry and FLOSS community to both academia and government;
65 64 \end{itemize}\\ \hline
66   -\end{tabular}%
67   -}
  65 +\end{tabularx}
68 66 \caption{Empirical SPB management method and its benefits}
69 67 \label{practices-table}
  68 +\end{tabular}%}
70 69 \end{table}
71 70  
72 71 \subsection{Project management and communication on the developing platform
... ... @@ -149,7 +148,7 @@ side, we also felt more encouraged to validate faster and give faster feedback
149 148 to the teams [..] We gave this feedback fast and they also gave quick feedback
150 149 for any our questions''}. The teams' synchronization was reinforced with the
151 150 implementation of a Continuous Delivery pipeline. The benefits of this approach
152   -were presented in our previous work \cite {?} and corroborate these research
  151 +were presented in our previous work \cite{siqueira2018cd} and corroborate these research
153 152 results. To 81.1\% of students and 75\% of senior developers, deploying new
154 153 versions of the SPB portal in production was a motivator during the project.
155 154  
... ...
icse2018/content/07-discussion.tex
... ... @@ -1,113 +0,0 @@
1   -\section{Discussion and Final Remarks}
2   -\label{sec:discussion}
3   -
4   -In this paper we examined the empirical model built in a collaborative project
5   -between government and academia that successfully harmonized the differences in
6   -the common approaches to software development management used by each side. We
7   -mapped the key decisions made over the 30-months of the project, that aimed to
8   -improve communication and the development process as a whole. We also elaborated
9   -two surveys and one interviews that were conducted separately for three groups
10   -of participants. We obtained a total of responses of 37 undergraduated
11   -students, eight IT market professionals, and two government officials. Finally,
12   -we collected post-mortem public data on project management carried out on the
13   -platform itself. The results revealed nine practices were developed from three
14   -main decisions taken and 11 benefits were obtained with the adoption of these
15   -practices.
16   -
17   -In our previous work \cite {meirelles2017spb}, we presented the unprecedent
18   -platform developed in the case study project and seven lessons learned taking into account only the
19   -academia-side view. The new results acquired in the current work corroborate
20   -with these lessons, adding the point of view of the government and the academia
21   -in diverse performed levels. In addition, these results suggest that many free
22   -software development practices can be replicated in other contexts in which the
23   -diversity and plurality of its stakeholders need to be leveled and reconciled.
24   -
25   -The results obtained also showed questions that were not overcome during the
26   -project and which we believe need to be evaluated for future collaborations
27   -between government and academia for software development:
28   -\begin {itemize}
29   -\item Improving understanding about collaboration: \textit{"During development,
30   -we realized that the development team also felt like the owner of the project,
31   -not just a mere executor. partnership, then it had a lot of that team issue to
32   -suggest things to be put into the project. It was not a customer relationship it
33   -was a partnership relationship, so there was a lot of issue suggesting by the
34   -team to be put into the project"}
35   -\item Discussion of roles and responsibilities: \textit{"Who had the power to
36   -make a decision? There was no one, because it was a very equal relationship. The
37   -two organs were on the same hierarchical level within the work plane. But this
38   -does not work well, you have to leave well defined to whom the last word belongs
39   -in the decisions, because the conflicts will always happen."}.
40   -\item Look for a balance in the requirements definition. The responses showed
41   -that the government felt that it was not detailed enough and the development
42   -team felt that the requirements needed to be matured with the use.
43   -\item Smoothing the intermediations between the different roles \textit{"When we
44   -had the [UnB] coordinator, when we forwarded a direct question to a developer,
45   -the coordinator responded. So that was negative, because we felt a little
46   -coerced from talking directly to the teams"}
47   -\end {itemize}
48   -
49   -As future work, we will reapply in another government-academia paternship
50   -project the practices evidenced in this case study, and conduct
51   -qualitative and quantitative research throughout its execution. We intend to
52   -prove the effectiveness in adopting free software development practices to
53   -align the demands and expectations of a G-A collaboration.
54   -
55   -\begin{comment}
56   -
57   -\begin{itemize}
58   -\setlength\itemsep{1em}
59   -\item \textit{How to involve students real-world projects, interacting with real customers}
60   -Em uma das perguntas abertas na pesquisa direcionada aos alunos, os respondentes abordaram a importância que o projeto teve na sua vida profissional: "O projeto me permitiu compreender um pouco sobre a dinâmica do mercado de software em Brasília e, em especial, dentro do serviço público.", "Ótima experiência. Agregou muito valor à minha formação acadêmica e profissional. Replico e repasso os conhecimentos adquiridos ainda hoje, mesmo com mais de um ano do término do projeto.", "Uma das características mais interessantes do projeto foi proporcionar aos alunos uma experiência de desenvolvimento de software muito similar ao que eles escontram em empresas. Isso inclui ter contato com clientes reais, planejar o ciclo de desenvolvimento e participação em times."; "Apesar dos cursos de graduação abordar teorias sobre desenvolvimento de software, a aplicação em um projeto real teve muito impacto positivo no meu desenvolvimento como profissional e me forneceu uma visão de mercado."
61   -
62   -\item \textit{The participation of experienced professionals is crucial to succcess of the project}
63   -Além dos resultados positivos sobre a importância da presença dos desenvolvedores seniors na visão dos alunos, em resposta aberta seus benefícios foram ratificados: ""Esse projeto foi essencial na minha formação. O contato prático com o mundo real e pessoas experientes, alinhado a metodologias flexíveis auto organizadas para o desenvolvimento de software proporcionou a produção de resultados rápidos e uma equipe bastante motivada, algo essencial para o sucesso de um projeto.", "Além de me permitir aprender com profissionais mais experientes e em diferentes áreas do processo de desenvolvimento de software."
64   -Este tópico também foi relatado do lado do governo: "Você tinha os revisores, que eram os desenvolvedores originais dos softwares, isso passava confiança e a gente sentia confiança no código."
65   -Por fim, os desenvolvedores senior relataram uma boa relação de aprendizagem com os alunos: "A relação com os alunos era respeitosa. Os alunos estavam abertos e parecia que se sentiam confortáveis para receber e dar orientações e sugestões."
66   -
67   -\item \textit{A balanced relationship between academia and industry}
68   -Um dos desenvolvedores trouxe a tona a prioridade dos compromissos escolares em relação ao projeto: "Dificuldades na comunicação presencial da equipe, por conta de diferentes horários dos alunos. Ausência de alunos em momentos importantes do projeto, por conta de provas e outras demandas universitárias."
69   -Alguns alunos abordaram o avanço profissional e técnico durante o projeto: "A equipe trabalhou em quase todos os momentos no que seria o estado da arte da Engenharia de Software", "O projeto como um todo trouxe bastante experiência técnica para quem estava iniciando no mundo da engenharia de software. Eu e outros colegas atingimos níveis de conhecimento que nos possibilitaram engajar em projetos de software de maneira muito mais impactante e confiante, participando das comunidades dos softwares utilizados e até nos tornando membros mais influentes, com permissão de realizar contribuições diretas no código oficial das plataformas, o que é algo que inspira e motiva ainda mais a colaboração entre membros da equipe e das comunidades de software livre envolvidas."
70   -
71   -\item \textit{Managing different organizational cultures}
72   -Os entrevistados do lado do governo retratam o problema de tradução entre os modelos adotados pelo lado do governo e pelo lado da academia: "Não era uma tarefa muito fácil, fazer este DE-PARA, porque a ferramenta de acompanhamento de projeto lá do MPOG é amarrada no modelo tradicional de gerenciamento de projetos", " A gente tinha que documentar, pedir um report para vocês [UnB]. Faziamos um processo de atualização meio que em conjunto, para chegar no equilíbrio pra levar pra reunião do projeto."
73   -
74   -\item \textit{Manage higher-level and lower-level goals separately}
75   -Não encontrei
76   -
77   -\item \textit{Living will ill-advised political decisions}
78   -Os funcionários do MPOG avaliam os impactos negativos de decisões técnicas tomadas com viés político: "Realmente, a questão do Colab foi o mais impactante [na complexidade] deu um impacto muito grande. Na prática, tomou essa decisão e gastou-se meses para poder fazer ele conversar com as outras ferramentas e esse tempo poderia ter sido gasto fazendo módulos novos, por exemplo, o que a gente tinha previsto, o Mercado Público, que é a questão dos prestadores de serviço lá dentro da ferramenta. Não conseguimos fazer. A gente teve que remover isso do escopo porque não teve não sobrou tempo. A gente gastou tanto esforço evoluindo o Colab (fazendo API, fazendo integração) que não sobrou tempo para fazer. Um negócio que tinham no portal antigo e no novo acabou que não foi feito.", Então ficar atento as premissas, detalhar melhor
79   -os requisitos e tentar o mínimo de influencia política num projeto. Porque essas influencias politicas, elas atrapalham demais, elas criam soluções insustentáveis. Esse caso aí do Colab não é simples. Colab, virou um exemplo, um exemplo de influências políticas que na hora pode ter feito sentido, mas depois gera muito overhead, dor de cabeça."
80   -
81   -\item \textit{Consider sustainability from the beginning}
82   -Durante a entrevista, um dos entrevistados do governo ressaltou a preocupação com sustentabilidade: "Porque você pode fazer o negócio mais perfeito possível, mas se você não manter là frente, você tem um problema. Se você não consegue manter, ali na frente você não vai ter uma equipe tão boa para manter, você vai ter um problema."
83   -\end{itemize}
84   -
85   -In addition to ratifying the lessons learned, some of the questionnaires or the
86   -interview answers raise negative points that need to be evaluated and better
87   -worked for future collaboration between government and academia. These negative
88   -responses are mainly related to differences in the management approaches of each
89   -organization.
90   -
91   -Do lado do governo:
92   -
93   -\begin{itemize}
94   -\item \textbf{Detalhamento dos requisitos}
95   -\subitem "Eu acho que é este detalhamento de requisito, eu acho que a gente tinha que exigir mais e melhor detalhamento dos requisitos."
96   -\subitem "Eu acho que, no final, a gente já tava numa liga legal no sentido de ter os requisitos num nível de detalhes necessários. Eu sempre achava que os requisitos não estavam documentados o suficiente pra a visão do cliente na hora do desenvolvimento. A gente desenvolvia, mas, às vezes, não era bem exatamente aquilo que o cliente queria."
97   -\subitem "Nem sempre quando o pedido de mudança partia da equipe isso era debatido ou acordado. Às vezes a gente só descobria lá no final da sprint. A gente tinha alguns probleminhas com isso. [..] Você precisa, sei lá, fazer uma entrevista, detalhar isso melhor, porque você vai codificar e você precisa saber. Ás vezes isso não acontecia e aí o programador fazia da forma que ele achava. Na hora da entrega, na hora de colocar no ar, a gente tinha algumas surpresas, algumas coisas que não estavam legais."
98   -\item \textbf{Discussão de papéis e responsabilidades}
99   -\subitem "Quem que tinha o poder de tomar uma decisão? Não tinha, porque era uma relação muito igual. Os dois órgãos ficavam no mesmo nível hierárquico dentro do plano de trabalho. Mas isso não funciona bem, você tem que deixar bem definido a quem pertence a última palavra nas decisões, porque os conflitos sempre vão acontecer."
100   -\item \textbf{Estabelecer limites sem coagir}
101   -\subitem "Quando tinha o coordenador (que coordenava as equipes), o que sentiamos de ruim era que, quando a gente encaminhava uma pergunta direto para um desenvolvedor, o coordenador respondia. Então isso era meio negativo, porque causava uma dificuldade, a gente se sentia meio coagido de conversar direto com as equipes.
102   -\end{itemize}
103   -
104   -Do lado da academia:
105   -
106   -\begin{itemize}
107   -\item bla
108   -\end{itemize}
109   -
110   -% "No final teve aquele medo, aquela pressão, que não saia recurso. Teve uma desmobilização grande no final porque não estava saindo recurso, teve que liberar os seniors, isso também atrapalhou. Então essa parte aí de política impacta bastante no nosso trabalho no que era entregue. Naquela época, as vezes até tinha o recurso, mas tinha uma resistência da alta administração, agora não tem o recurso."
111   -
112   -\end{comment}
113   -
icse2018/spb-oss-2018.bib
... ... @@ -216,3 +216,11 @@
216 216 booktitle = {Proceedings of the 2011 Eighth International Conference on Information Technology: New Generations}
217 217 }
218 218  
  219 +@inproceedings{siqueira2018cd,
  220 + author = {Siqueira, Rodrigo and Camarinha, Diego and Wen, Melissa and Meirelles, Paulo and Kon, Fabio},
  221 + title = {Continuous Delivery: Building Trust in a Large-Scale, Complex Government Organization},
  222 + booktitle = {Approved to IEEE Software - Special Issue: Release Engineering},
  223 + year = {2018},
  224 + organization={IEEE}
  225 +}
  226 +
... ...
icse2018/spb-oss-2018.tex
... ... @@ -37,10 +37,9 @@
37 37 \input{content/00-abstract}
38 38 \input{content/01-introduction}
39 39 \input{content/03-relatedwork}
40   -\input{content/04-case}
41   -\input{content/05-methods}
42   -\input{content/06-results}
43   -\input{content/07-discussion}
  40 +\input{content/04-methods}
  41 +\input{content/05-results}
  42 +\input{content/06-discussion}
44 43  
45 44 \bibliographystyle{splncs03}
46 45 \bibliography{spb-oss-2018}
... ...