Software Público Brasileiro

\section{Results}
\label{sec:results}

%TODO: Talvez esse paragráfo tem que está no Research Design
%%
The case study was analyzed and divided into two phases according to the project
management model. In the second phase (after one year of execution), several
practices have been applied to harmonize the cultural and organizational
divergences of the institutions involved.
%%
At the end of the project, an empirical
model of management and development process was created by aligning experiences
from the FLOSS universe, academic research and bureaucracies needed by the
government. In this section, we present by context the practices adopted in this
second phase and show the benefits generated by its deployment, as summarized 
in the Table \ref{practices-table}.

\begin{table}[]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{ | m{4cm} m{10cm} m{10cm} | } 
\hline
\textbf{Decision} & \textbf{Practice Explanation} & \textbf{Benefits} \\ \hline
\textbf{Project management and communication on the developing platform itself} 
& \begin{itemize} \item Migration of project management and communication into 
the platform under development using its integrated software components Gitlab 
and Mailman \end{itemize} & \begin{itemize} \item Confidence in developed code; 
\item Transparency and efficiency in communication; \item Easier monitoring and 
increase interactions between development team and public servants; \item 
Organically documentation and records generation; \end{itemize} \\ \hline
\textbf{Bring together government staff and development team} & \begin{itemize} 
\item Biweekly gov staff, senior developers and coaches met to planning and 
review sprint at the UnB headquarters. \item Most of features under development 
were discussed on Gitlab Issue Tracker. \item Only strategic decisions or 
bureaucratic issues involve board directors. \item Deploying SPB intermediated
versions in production \end{itemize} & \begin{itemize} 
\item Reducing communication misunderstood; \item Empathy between members on 
both sides; Meeting expectations of both sides on developing requirements; 
\item Improving the understanding of collaborative development by the 
government; \item Aligning both side pace to execute project-related 
activities; \item 
Increasing government confidence for collaborative projects with the 
university; \item Motivating teams; \item 
Aligning both side pace to execute project-related activities; \item Improving 
translation from one development process to the other; \end{itemize} \\ \hline
%\textbf{Continuous Delivery} & 
%\item At each release, DevOps members would go to the ministry to assist its 
%infrastructure team in deploying. \end{itemize} & \begin{itemize} \item 
%Increasing government confidence for collaborative projects with the 
%university; \item Motivating teams; \item Transfering of knowledge about DevOps 
%and Continuous Deliveries from academia to gov infrastructure team; \item 
%Aligning both side pace to execute project-related activities; \item Improving 
%translation from one development process to the other; \end{itemize}\\ \hline
\textbf{Divide development team in "component" fronts} & \begin{itemize} \item 
The development was divided into four fronts with a certain self-organization 
of tasks. \item IT market professionals with recognized experience on each 
front were hired to work in person or remotely. \item For each front, there was 
at least one senior developer and the role of coach. \item The meta-coach role 
was also defined to coordinate tasks between teams. \end{itemize} & 
\begin{itemize} \item Helping conciliation of development processes and 
decision-making; \item Creating support-points for students, senior developers, 
and gov staff; \item Transfering of knowledge from industry and FLOSS community 
to both academia and government; \end{itemize}\\ \hline
\end{tabular}%
}
\caption{Empirical SPB management method and its benefits}
\label{practices-table}
\end{table}

\subsection{Project management and communication on the developing platform
itself}
\hfill

After nine months of project activities, the first version of new SPB Portal was
released. From this point, we started to migrate the management and 
communication interactions to the platform under development. In short, Wiki 
feature was used for meeting logging, defining goals, sprint planning, and 
documentation of deployment processes and administration resources guide. Issue 
tracker was used for discussing requirements, monitoring the features under 
development, registering changes, and validating functionalities delivered. The 
whole team used Mailing list to defining schedules of meetings and deliveries 
and also to collaborative definition of requirements.

Our surveys report a \textbf{transparency and efficiency in communication}. 
Senior developers and students used mostly via Mailing list (100\%) and Issue 
Tracker (62.5\%). Developers and MPOG staff interacted mostly via Mailing List 
(87.5\%) and Issue tracker (50\%). For example, a MPOG IT analyst said that the 
"communication goes far beyond that, you communicate to everyone in the project 
everything that was happening. We did not use emails, we use the mailing list 
more and avoid e-mails, it helped a lot because everything was public and did 
not pollute our mailbox. You wanted to know something, could go there and look 
at what was happening."

Migrating to SPB platform also provided an \textbf{easier monitoring and 
increase interactions between development team and public servants by 
coordinators}. As shown by collected data, 775 issues and 4,658 issue comments 
was registered during the project in the main repository (without considering 
the software repositories that integrated the platform) within the SPB 
platform. The issues have 59 different authors (8 from MPOG staff), and 
commented by 64 different users (9 form MPOG staff and users). Considering the 
most active issues those that have 10 or more comments, in a set of 84 issues, 
MPOG staff authored 36 issue (about 43\% of 84 issues with higher level of 
interaction). An MPOG analyst highlighted that “there was a lot of evolution, 
a lot of communication via Gitlab". This interaction also led MPOG staff to 
\textbf{confidence in developed code}: "Everything was validated, we tested the 
features and the project was developed inside the platform, so that the feature 
was validated in the development of the software itself. [..] From the moment 
we installed it, and began to use it for development, this validation was 
constant. We felt confident about the features".

One of the main concerns of traditional approach is meticulous documentation of 
software and development steps. With this aforementioned practice, we could 
meet this government demand without bureaucracies and changes in our 
development process, generating \textbf{organically documentation and records 
generation} in the platform itself, as one of the MPOG response evidenced: "For 
me it was a lot of learning, there is a lot of things documented in the e-mails 
and also there in the portal itself of what happened in the project. At any 
moment we can go there and see how it worked, how the person did, and manages 
to salvage those good points."

%\subsubsection{Bringing the government staff directly responsible for the 
%project together with development team}
%\begin{itemize}
%\item \textit{Biweekly meetings (planning and sprint review) in the 
%development lab with the presence of government staff, team coaches and senior 
%developers}
%\item \textit{Discuss features under development directly on Gitlab Issue 
%Tracker}
%\item \textit{Only strategic decisions or bureaucratic issues involve the 
%directors/secretaries}
%\end{itemize}
%
%\paragraph{Benefits}
%
%\begin{itemize}
%  \setlength\itemsep{1em}
%  \item \textit{Reduce communication misunderstood}
%    \subitem MPOG: "That's when the project started, people [MPOG staff] did 
%not participate in anything. The communication process was horrible."; "The 
%[MPOG] coordinator did not help, he would say something and UnB would talk to 
%another at the meeting and it was the biggest mess." About the direct dialogue 
%between the academic team and MPOG staff (without the involvement of 
%coordinators and / or directors) , she said "That's where things really started 
%to move, that the communication of the project began to improve."
%%
%  \item \textit{Empathy between members on both sides}
%    \subitem {72.9\%} of students believe that interacting with MPOG staff was 
%important during the project
%    \subitem Only 27\% of the students interviewed said they did not feel like 
%attending meetings with MPOG employees
%    \subitem MPOG: "You know people in person and it makes such a big 
%difference because it causes empathy. You know what the person is going through 
%on their side and she knows what we're going through on our side. So the next 
%time you have a non-personal interaction (by mail, by list ...) I think it even 
%facilitates, improves communication. You already know who that person is, it's 
%not just a name. "
%%
%  \item \textit{Develop requirements closer to the expectations of both sides}
%    \subitem {81.1 \%} of students believe that the participation of MPOG 
%staff in planning and closing sprints was important for the development of the 
%project
%    \subitem {75.6 \%} of students believe that writing the requirements 
%together with the MPOG staff was very important
%    \subitem Undergrad student: "Joint planning and timely meetings were very 
%important for understanding the needs of MPOG, and the interaction via SPB 
%tools helped validate the tool as a development platform"
%    \subitem Undergrad student: "Often they did not know what they really 
%wanted, and they caused some delays in the development of sprints"
%    \subitem Undergrad student: "A relationship of constant attempt to balance 
%and negotiate. The client does not always know the impacts of their requests"
%    \subitem MPOG: "I believe it was very positive, we also liked to go there, 
%to interact with the team. I think it brought more unity, more integration into 
%the project, because we went there, where people were working and they show 
%what was done. I think they also liked to receive our feedback about what had 
%been done by them.This interaction did not just made with the coordinator. I 
%found it very important and very positive it. "
%%
%  \item \textit{Improve understanding of collaborative development by MPOG 
%staff}
%    \subitem Undergrad student: "In the beginning the demands of MPOG were 
%very 'orders from above', but according to the progress of the project, they 
%understood better our work philosophy and became more open"
%    \subitem MPOG: "During development we realized that the team that was 
%developing also felt like the owner of the project felt involved not only a 
%mere executor of an order. It was not a client relationship, it was a 
%partnership relationship, so there was a lot of team suggestions to be put into 
%the project. Sometimes these were put in for us to decide and sometimes not."
%    \subitem MPOG: "I think it was easy, I think the team was aligned. In 
%addition to being aligned, these items that, for example, were not priorities 
%and became priorities, were, in a sense, brought with some arguments from the 
%team. So the team was able to argue and succeed in showing that it was 
%important, that it needed to be prioritized, and I think the team was able to 
%present the arguments well for some of the priorities that happened during the 
%process."
%%
%  \item \textit{Align the pace of both sides to execute activities}
%    \subitem MPOG: "When we went there, I knew people and made that 
%interaction more frequent, we also felt encouraged to validate faster and give 
%faster feedback to the teams so they would not wait there. I knew they were 
%waiting for our feedback and we were struggling to do it fast, because we ended 
%a sprint and start another and not stop. We gave that feedback fast and they 
%also gave quick feedback for any questions when they encountered a problem. 
%That gave the project agility, things flowed faster and better. "
%\end{itemize}

%\subsubsection{Continuos Delivery}
%
%\begin{itemize}
% \item \textit{Creating DevOps Team}
% \item \textit{Defining continuous delivery pipeline}
% \item \textit{DevOps team periodically going to the ministry to help deploy 
%each version}
%\end{itemize}
%
%\paragraph{Benefits}
%
%\begin{itemize}
%  \setlength\itemsep{1em}
%  \item \textit{Increase government confidence for collaborative projects with 
%the
%university}
%    \subitem MPOG: "At first the government staff had a bias that universities 
%did
%not deliver and we overcame that bias in the course of the project. We deliver 
%a
%lot and with quality. Today, I think that if we had paid the same amount for a
%company, it would not have done what was delivered and with the quality that 
%was
%delivered with the price that was paid."
%  \item \textit{Motivate teams}
%    \subitem {81.1\%} of students think new versions released in production 
%motivated
%them during the project
%    \subitem {75\%} of senior developers think new versions released in 
%production
%motivated them during the project
%    \subitem {81\%} of students think the presence of a specific DevOps team 
%was
%necessary for the project
%  \item \textit{Transfer of knowledge about DevOps and Continuous Deliveries 
%from
%the academic team to the government infrastructure team}
%    \subitem MPOG: "I only noticed positive aspects in the delivery. I think 
%in the
%interaction, we had a lot of support to be able to deploy. From the time that
%the version was mature, which had already been tested in the UnB test
%environment and was ready to be put into production, we had a great agility to
%release in production. Then in the course of the project we realized that the
%infrastructure team [of MPOG] started to trust the UnB team a lot. Because, for
%you to put software in production in government, there is a whole process
%behind. The government has much of this security issue."
%    \subitem MPOG: "If there was anything stopping the business from working, 
%the
%software working inside, we would ask the seniors for support so we could
%investigate that, and the infrastructure team was also instructed to prioritize
%it. So when it came to an impasse, the teams were all together, both from 
%within
%MPOG as well as senior developers and other UnB developers to unlock, to find
%the problem."
%  \item \textit{Align the university and government teams pace in the 
%execution of
%the activities}
%    \subitem MPOG: "In the beginning, infrastructure personnel were not 
%accustomed
%to deliveries so fast. They had to adapt to this pace. The portal of the SPB
%before the project was not there [in the MPOG infrastructure], it was in 
%another
%place, they did not have that dynamics there. But what they asked for UnB (some
%configuration, installation manual, how to install everything inside) was
%requested and delivered."
%  \item \textit{Improve translation from one development process to the other}
%    \subitem MPOG: "We had an overview at the strategic level, but when we 
%went down
%to the level of functionality we had this difficulty to do the planning of the
%release every four months. But in the end, I think this has not been a problem,
%because a project you are delivering, the results are going to production, the
%code is quality, the team is qualified/capable and the project is doing well, 
%it
%does not impact as much in practice, because the result is being delivered.
%\end{itemize}
%
%\subsubsection{Organization of the project in teams for each front, with a
%undergraduate student as coach and at least one senior developer}
%
%\begin{itemize}
%  \item \textit{Four fronts: Colab, Noosfero, DevOps and Front-End/UX}
%  \item \textit{Definition of the role of team coaches and meta-coach, 
%selected from undergraduate students group}
%  \item \textit{Hiring professionals from the IT market for face-to-face or 
%remote work, specialists in the software components}
%\end{itemize}
%
%\paragraph{Benefits}
%
%\begin{itemize}
%  \setlength\itemsep{1em}
%  \item \textit{Help to conciliate development processes and decision-making}
%    \subitem {62,5\%} of senior developers believe they have helped MPOG staff 
%to more clearly express their requests
%    \subitem {87,5\%} of seniors agreed with the project development process. 
%For 37.5\% this process was little similar to their previous experiences, for 
%the others there was a certain similarity.
%    \subitem {62,5\%} of seniors did not understand MPOG's project management 
%process. {50\%} of them believe their project productivity was affected by 
%MPOG's project management process.
%    \subitem Senior Dev: "I think my main contribution was to have balanced 
%the relations between the MPOG staff and the UnB team"
%    \subitem Senior Dev: "When I entered the project, the client had a 
%disproportionate view of how to make explicit the requirements. They were still 
%talking about use cases and were extremely concerned about validation processes 
%and acceptance of these documents."
%    \subitem MPOG: "You had the reviewers, who were the original developers of 
%the software, that gave you confidence and confidence in the code."
%%
%  \item \textit{Create support and reference points for students, senior 
%developers, and government staff}
%    \subitem {89.1\%} of students believe that the presence of the leader was 
%essential to the running of Sprint
%    \subitem {87.5\%} of seniors believe that the presence of team leaders was 
%essential for their interaction with the team
%    \subitem MPOG: "It interacted more with the project coordinator and team 
%coaches (noosfero, colab, visual identity). Interacted with coaches by mailing 
%list, hangouts The reason was usually to elucidate requirements, to ask 
%questions about requirements, to understand some functionality. "
%    \subitem MPOG: "There was interaction with the other [non-coaches] because 
%they also participated in the bi-weekly meetings (sprints), but it was more 
%with coaches."
%    \subitem MPOG: "Access to coaches was faster, because we were in much more 
%interaction with leaders than with senior developers. Sometimes the coaches 
%brought the question to the senior developers."
%%
%  \item \textit{Transfer of knowledge from industry and FLOSS community to 
%both academia and government}
%    \subitem {62.5\%} of senior developers believe that they have collaborated 
%in the relationship between the management and development processes of the two 
%institutions
%    \subitem {100\%} of the students we interviewed believe that working with 
%senior developers was important during the project
%    \subitem {91.\%} of students also believe that working with seniors was 
%important for learning
%    \subitem {75\%} of senior developers believe that 'Working in pairs with a 
%senior' and 62.5% who 'Participate in joint review tasks' were the tasks with 
%the involvement of them that most contributed to the evolution of students in 
%the project.
%    \subitem {75\%} of senior developers believe that in guiding a student, 
%this knowledge was widespread among the other students on the team.
%    \subitem MPOG: "On the side of UnB, what we perceived so strongly was that 
%the project took a very big leap when the original developers of the software 
%(the official software development) were hired in the case of Noosfero and 
%Colab [..] Because they had a guide on how to develop things in the best way 
%and were able to solve non-trivial problems and quickly "
%\end{itemize}
%
%%* Filtrar a comunicação por níveis de maturidade/experiência e 
%responsabilidades
%%MPOG: "Eu acho que esses pontos de conflito eram muito mais fáceis de lidar 
%com a equipe do que com a própria coordenação. [..] Eu acho que tem uma 
%diferença também de papel tem uma diferença de postura. Eu acho que a 
%relação com a equipe, embora ela fosse saudável, eu acho que a equipe não 
%tinha tanta autonomia quanto à coordenação tinha. Então talvez fosse mais 
%difícil com a coordenação e não com a equipe, porque a equipe ela sabia o 
%limite dela e a partir dali ela não agia mais, ela já convocava a 
%coordenação para lidar (a gerência)."