Merge branch 'master' of http://softwarepublico.gov.br/gitlab/softwarepublico/articles

Paulo Meireles
2 parents fadd8b51 e531f1c0
Showing 2 changed files with 135 additions and 127 deletions Show diff stats
opensym2017/content/05-features.tex
opensym2017/content/07-process.tex
 \section{Features}
 \label{sec:spb}
-The new generation of the SPB portal combines adapted features of existing collaborative softwares
+The new generation of the SPB portal combines features adapted from existing collaborative software
 and features developed by the SPB team. Whenever possible, new functions 
-(newly developed or partially modified) were sent to official repositories, as a contribution.
+(newly developed or partially modified) were contributed back to the official repositories.
 As a result, we have a platform that integrates and harmonizes different features such as
 social networking, mailing list, version control system, content management and
 source code quality monitoring. Our aim was to develop functionalities by reusing
-functions of collaborative softwares already integrated to the platform. In
+functionality of collaborative software already integrated to the platform. In
 addition, we tried to keep this integration transparent to end users.
 \subsection{Software and Software Community}
@@ -17,47 +17,47 @@ In the new SPB portal, each software has a standard set of pages and tools.
 Besides accessing support pages (such as FAQ and installation guide) within the platform, users will be able to download
 different versions of the software and find several mechanisms of software development management.
-Focusing on the collaborative development, the Mailman was integrated to the platform in order to allow
+Focusing on the collaborative development, Mailman was integrated to the platform in order to allow
 the dialogue and communication between developers, users and
 enthusiasts of a determined software. The software has its own mailing list whose privacy
 can be configured/set by administrators. 
-The software has a social interface area (aka "software community") where users can find other users, blogs,
+The software has a social interface area ("software community") where users can find other users, blogs,
 summary of recent activities, or any other relevant community-produced content. 
 Users logged to the platform can request membership to different software communities 
 and each community member can access and edit restricted content. For this purpose, 
-many Noosfero features related to social network and content management was integrated to the portal.
+many Noosfero features related to social networking and content management were integrated to the portal.
 To assist decision-making, the new SPB has acquired assessment and statistical
 tools. Now, users will be able to rate the software and make comments and all
 information will be avaiable to other users. Moreover, the software has a section
-containing  its statistical data, where values are calculated through data
+containing  its statistical data, where values are calculated based on data
 provided by users and the system.
 The role of the administrator will be present in the software and in its community. The
 administrator is responsible for moderating content, memberships and user
-comments. He is also the one who can make changes in the software homepage
+comments. The administrator is also the one who can make changes in the software homepage
 content.
 \subsection{Software Catalog and global search}
 The platform also provides a search tool called Software Catalog, 
-which allows users to find softwares available in its directory.
+which allows users to find softwares available in the portal.
 In this catalog, some search options were developed to make the navigation easier,
 such as filters (by type of software or category), sorting and score.
 In order to expand the searching scope and cover more types of content, the SPB team
 developed the global search tool. This tool unifies search mechanisms
-provided by the mentioned collaborative softwares. Any user can
-find a public content in the context of social network, mailing list and
+provided by the different collaborative software used in the SPB protal. Any user can
+find a public content in the context of social networking, mailing list and
 software development.
 \subsection{Software development tools}
 The new SPB also provides
-tools to encourage developers to keep each source code and its
-developments within the platform. Any created software has, by default, a
-related git repository with wiki pages and issues tracking. These tools are
+tools to encourage developers to keep source code and its
+development activity within the platform. Any created software has, by default, a
+an associated git repository with wiki pages and issue tracking. These tools are
 supplied by the integration of Gitlab into the platform. 
 Developers can also evaluate the software source code to measure software
@@ -67,7 +67,7 @@ public, which allows greater transparency between the developer and the
 community that uses the software.  Thereby, the maintainers can decide if the
 given solution meets the source code quality requirements.
-Thus, the SPB becomes a platform to stimulate the openness of the source code;
+Thus, the SPB becamr a platform to stimulate the openness of the source code;
 dialogue between users and the development team; and also
 maintenance and evolution of the software, which will provide more
 transparency in government investments.
@@ -5,19 +5,19 @@ The SPB team was composed of a variety of professionals with different levels
 and skills, where most of them were undergraduate students with major in
 software engineering (from 4th semester or upper).  Since the students could
 not dedicate many hours per week to the project, they always had the
-flexibility to negotiate their work schedule during the semester in order not
-to cause any damage to their grades. Their daily work routine in the project
-included programming and devops tasks. 
+flexibility to negotiate their work schedule during the semester in
+order to not harm their classes and coursework. Their daily work routine
+in the project included programming and devops tasks.
 The development of SPB project required a vast experience and background that
-usually undergraduate students do not have yet. For this reason, some senior
-developers have joined to the project to help with hard issues and to transfer
+usually undergraduate students do not have yet. For this reason, a few senior
+developers have joined to the project to help with the more difficult issues and to transfer
 knowledge to the students. Their main task was to provide solutions for complex
-problems, in other words, they worked as a developer. As these professionals
+problems, in other words, they worked as developers. As these professionals
 are very skillful and the project could not fund a full time work for them,
 some of them worked partially on the project. In addition, they lived in a
 different states spread around Brazil which led much of the communication to be
-made via Internet.
+online.
 In short, our work process was based on open and collaborative software
 development practices. The development process was defined based on the
@@ -26,70 +26,104 @@ high degree of automation resulting from DevOps practices. Thus, the work
 process was executed in a cadenced and continuous way.
 Finally, the last group of actors of this project was composed of employees
-formally working for the Brazilian government, in the Ministery of Planning,
-Development, and Management (MP is the Brazilian acronyms). All the project
+formally working for the Brazilian government, in the Ministry of Planning,
+Development, and Management (MP is the Brazilian acronym). All the project
 decisions, validations, and scope definitions were made by them. In this way we
-developed software product increments, releases,  aligned with business
-strategic objectives. As can be seen, the project had many kinds of profiles
-that had to be organized and synchronized.
+developed software product incrementally, with releases aligned to business
+strategic objectives. As one can see, the project had many different
+kinds of stakeholders that had to be organized and synchronized.
-\subsection{Teams organizations}
+\subsection{Team organization}
 Approximately 70\% of the development teams were composed of software
-engineering undergraduate students from UnB and they worked physically in the
-same laboratory in the opposite of the senior. Each student had their own
-scheduler based on their class, it made complicated to implement pair
-programming. Also, they had a different area of interests. To cope with those
-diversity, we had two basic rules which guided the project organization:
+engineering undergraduate students from UnB and they worked physically
+in the same laboratory. Each student had their own schedule based on
+their classes, what complicates the implementation of pair programming.
+The senior developers tried to synchronize their schedule with the
+students on their sub-team. To  cope with this environment, we had a few
+basic rules which guided the project organization:
 \begin{enumerate}
-  \item Classes have to be the high priority for undergraduate students;
-  \item Always work in pair (locally or remotely).
+  \item Classes have high priority for undergraduate students;
+  \item Work in pairs whenever possible (locally or remotely).
+  \item There must be one morning or afternoon per week when
+    \emph{everyone} should be together physically in the laboratory
+    (except of course the remote team members).
+  \item Every 3 to 4 months, the senior developers would fly in and work
+    alongside the students for a few days.
 \end{enumerate}
-With the aforementioned rules we divided all the project into four different
-teams: Colab, Noosfero, Design, and DevOps. Each team had one coach responsible
-for reducing the communication problem with the other teams and help the
-members to organize itself in the best way for everyone (always respecting the
-work time). The coach, was a normal student working in some of the teams with
-the extra duty of register the current tasks developed in the sprint and with
-the responsibility to talk with other teams. One important thing to notice is
-the mutability of the team and the coach, during the project many students
-changed between the teams to try different areas.
-
-One characteristic of the teams was the presence of (at least) one senior per
-team. This was essential, because hard decisions and complex problems were
-usually addressed to them, this relieved the coach duty to take a complicated
-technical decisions and encouraged students to be a coach. Lastly, the senior
-had to respect a rule number two and work with students, this was important to
-gave the undergraduate the opportunity to interact with a savvy professional in
-his area and keeping the knowledge flow in the project.
-
-Finally, we had to add two last elements of the team organization, that was
-essential for the project harmony: the meta-coach and professors. The former
-was a software engineer recently graduated and which wanted to keep working on
-the project, the latter were professors that orchestrated all the interactions
-between all members of the project. The meta-coach usually worked in one
-specific team and had the extra task of knowing the current status of all
-teams. Professors and meta-coaches worked together to reduce the communication
-problem between all the teams. Lastly, all the bureaucracy tasks was
-centralized in the professors.
-
-\subsection{Meetings}
-
-Brazilian government used to work with software development in a very
-traditional way, frequently they claim on documents and does not focus on what
-really matter (running software). This way of thinking caused to us a
-communication noise with MP, because they constantly tried to leverage on our
-work style. It was especially hard to convince them to accept the idea of open
-scope and agile development, but after months of labor and showing results they
-stopped resisting.
-
-We defined some level of meeting granularity to avoid to generate overheads to
-the developers. We had a strategical and validating meeting with MP (the
-former once in a month and the latter each 15th day), release plaining with the
-entire team (one per month), and finally a sprint planning (one each 15th day).
-Figure \ref{fig:meeting} is a diagram that represents our meeting organization.
+With the aforementioned rules we divided all the project into four
+different teams: Colab, Noosfero, Design, and DevOps. Each team had one
+coach responsible for reducing the communication problem with the other
+teams and help the members to organize themselves in the best way for
+everyone (always respecting their work time). The coach was one of the
+students working in some of the teams with the extra duty of registering
+the current tasks developed in the sprint and with the responsibility to
+talk with other teams. One important thing to notice is the mutability
+of the team and the coach, during the project many students changed
+between the teams to try different areas.
+
+One characteristic of the teams was the presence of (at least) one
+senior per team. This was essential, because hard decisions and complex
+problems were usually referred to them. This kept having to take
+complicated technical decisions from the coach tole, what encouraged
+students to be coaches. Lastly, the senior developers worked directly
+with the students, and this was important to give the undergraduate the
+opportunity to interact with a savvy professional in his area and to
+keep the knowledge flowing in the project.
+
+Finally, we had to add two last elements of the team organization, that
+was essential for the project harmony: the meta-coach and professors.
+The former was a software engineer recently graduated and which wanted
+to keep working on the project, the latter were professors that
+orchestrated all the interactions between all members of the project.
+The meta-coach usually worked in one specific team and had the extra
+task of knowing the current status of all teams. Professors and
+meta-coaches worked together to reduce the communication problem between
+all the teams. Lastly, all the paperwork tasks, such as reporting on the
+project progress to the Ministry, was handled by the professors.
+
+\subsection{Communication and management}
+
+Our team had many people working together, and most of the seniors worked in a
+different city remotely. Also, we tried to keep our work completely clear to
+the Brazilian government and citizens interested in following the project. To
+handle those cases, we used a set of tools to communication and other to manage
+the project.
+
+For communication between member in different places, we used: Google
+handouts with tmate, IRC, and mailing lists. When one student had to
+work in pair with a senior, normally, they used Google hangout for
+communication and they shared a terminal session with tmate which allow
+them to share the same terminal, with both typing and seeing the screen.
+For questions and fast discussion, we used IRC. For general
+notification, we used the mailing lists.
+
+For managing the project we used the SPB Portal itself; first to
+validate it by ourselves, and also because it had all the required
+tools. We basically created one wiki page per release Gitlab, one
+milestone per sprint, and one or more issues for addressing each
+feature. With this approach we achieved two important goals: keeping all
+the management as close possible to to the source codem, and tracking
+every feature developed during the project.
+
+\subsection{High-level project management and reporting}
+
+The Brazilian government used to work with software development in a
+very traditional way. They would frequently focus on documents and not
+on what was, in our opinion, what really matters: working software. This
+dissonance caused us a communication noise with MP, because they would
+often question our work style.  It was especially hard to convince them
+to accept the idea of open scope and agile development, but after months
+of labor and showing results they stopped resisting.
+
+We defined some level of meeting granularity to avoid generating too
+much overhead to the developers. We had a strategical and validating
+meeting with MP (the former once in a month and the latter each 15th
+day), release plaining with the entire team (one per month), and finally
+a sprint planning (one each 15th day).  Figure \ref{fig:meeting} is a
+diagram that represents our meeting organization.
 \begin{figure}[hbt]
   \centering
@@ -98,36 +132,36 @@ Figure \ref{fig:meeting} is a diagram that represents our meeting organization.
   \label{fig:meeting}
 \end{figure}
-In the strategical meeting we usually defined the priorities and new features
-with the Brazilian government (we always had to negotiate next steps with
-them). Normally the professors, the coach of each team, the meta-coach, and
-some employees of the MP join in this meeting. We usually discussed what the
-team already produced since our last meeting, and we establish the new features
-for the next release.  Notice that just part of the team join in this meeting
-to avoid generating unnecessary overhead to the developers, but all the
-students interested to participate was allowed to join (many students wanted
-this experience during the project).
+In the strategical meeting we usually defined the priorities and new
+features with the Brazilian government. Normally the professors, the
+coach of each team, the meta-coach, and some employees of the MP would
+participate in this meeting. We usually discussed what the team already
+produced since our last meeting, and established the new features for
+the next release.  Notice that just part of the team would join this
+meeting to avoid generating unnecessary overhead to the developers, but
+all the students interested to participate were allowed to join (many
+students wanted this experience during the project).
 After the strategical meeting with Brazilian government agents, we had a
 planning turn with all teams together. In this part, each team worked together
-to convert the MP wishes into small parts which was represented by the epics of
+to convert the MP wishes into smaller parts which were represented by the epics of
 the release. Each coach was responsible for conducting the planning, and after
-that register it on the project wiki (the wiki provided by Gitlab). With this
+that record it on the project wiki (the wiki provided by Gitlab). With this
 epic, each 14th day the team have generated their sprint scheduler (with small
-achievements mapped in issues).
+achievements mapped to issues).
-To keep the Brazilian government always updated, we invited them to work with
-us to validate the new features in progress. Normally we had a meeting each
-15th day. Basically, this was our work flow, we always kept everything
-extremely open to the MP (our way of work and open source projects) and to the
-team.
+To keep the Brazilian government always updated, we invited them to work
+with us to validate the new features in progress. Normally we had a
+meeting each 15th day. Basically, this was our work flow, we always kept
+everything extremely open to the MP (our way of working, and the one
+often used by open source projects) and to the team.
-To keep the track of all of those things we used the SPB, especially the
-Gitlab. Basically, we had:
+To keep the track of all of those things we used the SPB itself,
+especially Gitlab. Basically, we had:
 \begin{enumerate}
   \item Project repository: We have one organization with many repositories
-  \item Milestones: Each milestone is used to register a release
+  \item Milestones: Each milestone was used to register a release
   \item Wiki: Each release has one page on wiki with the compilation of
     strategical meeting
   \item Issues: Each sprint planning generated issues, which we associated to
@@ -135,32 +169,6 @@ Gitlab. Basically, we had:
     with them. Finally each developer assigned the issue to itself.
 \end{enumerate}
-Notice that this workflow gave to us and to the Brazilian government agents a
-full traceability from high view of the feature to the low view (code). This
-provided to them a way to  validate all worked done and proof the concept that
-work with open source project can give a proper view to them check.
-
-\subsection{Tools for communication and management}
-
-Our team had many people worked together, and most of the seniors worked in a
-different city remotely. Also, we tried to keep our work completely clear to
-the Brazilian government and citizens interested in follow the project. To
-handle those cases, we used a set of tools to communication and other to manage
-the project.
-
-For communication between member in different places, we used: google-talk with
-tmate, IRC, and mailing-list. When one student had to work in pair with a
-senior, normally, they used google-hangout for communication and they shared a
-session with tmate which allow them to share the same terminal. For questions
-and fast discussion, we used IRC. For general notification, we used the
-mailing-list.
-
-For managing the project we used the SPB Portal to validate it by ourselves and
-because it had all the required tools. We basically create one wiki page per
-release in Gitlab, one milestone per sprint, and one or more issues for address
-one user history. With this approach we achieve two important things: keep all
-the management close to the source code and tracked every feature developed by
-the project.
-
-%TODO: Ainda falta adicionar a parte da visita dos seniors e o turno sagrado
-
+Notice that this workflow gave us and the Brazilian government agents
+full traceability from a high level view of each feature to the lowest
+level (code).