diff --git a/OSS-2017/00-abstract.tex b/OSS-2017/00-abstract.tex index 7d07acb..2d030c4 100644 --- a/OSS-2017/00-abstract.tex +++ b/OSS-2017/00-abstract.tex @@ -1,10 +1,9 @@ \begin{abstract} -Brazilian Public Softwares have some differences from Open Source Softwares, -in particular the software is considered a public good and the Federal -Government assumes some responsibilities related to its use. Anyway, the -software development principles are the same: trend towards decentralization in -decision-making, information and development sharing, and feedback. In this -context, we have designed a platform based on the integration and evolution of -existing OSS tools. Nowadays, the Brazilian Public Software Portal provides -several modern features for software collaborative development. +The Brazilian Public Software is a program by the Brazilian Federal +Government to foster the sharing and collaboration on FOSS solutions for +the public administration. In this paper, we present an integrated +development platform that was developed for the program. The development +of this platform used several FOSS applications, and contributed +features back to them. We discuss the development methodology, platform +architecture, and the user experience efforts carried out. \end{abstract} diff --git a/OSS-2017/01-introduction.tex b/OSS-2017/01-introduction.tex index 0239901..37f59a2 100644 --- a/OSS-2017/01-introduction.tex +++ b/OSS-2017/01-introduction.tex @@ -1,25 +1,50 @@ \section{Introduction} \label{sec:intro} -Since last few decades, the Brazilian Federal Government has been improving its -software adoption and development processes. In 2003, the recommendation to -adopt Open Source Software (OSS) become a public policy. In 2007, the Brazilian -Government released a Portal to share projects like OSS called Brazilian Public -Software (\textit{Software Público Brasileiro} -- SPB). In short, it is a -website to share softwares developed by and for the Brazilian Government. - -On the one hand, the Brazilian legal instrument on software contracting -(\textit{Instrução Normativa} 04/2012) indicates that public managers must -consult the SPB Portal to adopt a software solution, as well as, justifying the -acquisition of a proprietary software if there is no a similar project -available in the SPB Portal. On the other hand, since 2009, the SPB Portal had -several technical problems because there was no development activities to -maintenance and evolve it. Thus, the initial SPB Portal version did not have -another release. - -From January 2014 until June 2016, a platform for the SPB Portal was designed -and developed by the University of Brasília (UnB) and the University of São -Paulo (USP). This new Portal was designed to be an integrated platform of -software collaborative development environments with social networking, mailing -list, control version system, and source code quality monitoring. In this -paper, we present this new generation of the SPB Portal. +During the last few decades, the Brazilian Federal Government has +improved its software adoption and development processes. In 2003, the +recommendation to adopt Free/Open Source Software (FOSS) become a public +policy. In 2007, the Brazilian Government released a portal called +Brazilian Public Software (\textit{Software Público Brasileiro} -- SPB, +in Portuguese), with the goal of sharing FOSS projects developed by, or +for, the Brazilian Government. + +The Brazilian legal instrument on software contracting +(\textit{Instrução Normativa} 04/2012) mandates that public management +must consult the SPB Portal to adopt a software solution. The +acquisition of a proprietary solution must be explicitly justified by +demonstrating that there is no suitable option in the SPB Portal. + +Since 2009, however, the SPB Portal was having several technical issues. +The original codebase was not being developed anymore, and there as a +large amount of technical debt to overcome. The system was a modified +version of an existing FOSS platform that was not being developed +anymore, and the portal maintenance was becoming harder and harder. + +From January 2014 to June 2016, a new platform for the SPB Portal was +designed and developed by the University of Brasília (UnB) and the +University of São Paulo (USP) in a partnership with the Brazilian +Ministry of Budget, Planning, and Management. This new Portal was +designed as an integrated platform for collaborative software +development. It includes functionality for social networking, mailing +lists, version control system, and source code quality monitoring. In +this paper, we present an overview of this new generation of the SPB +Portal. + +The project was developed by a team of 3 professors, 6 professionals, 2 +masters students, and approximately 40 undergrad students (not all of +them at the same time, though -- graduations and other events triggered +changes in the team). + +Figure \ref{fig:spb} shows the home page of this integrated platform. +The development tried to be as faithful as possible to FOSS development. +All development was done in the open, and the changes we needed in the +tools user were contributed back to their communities. + +\begin{figure}[hbt] + \centering + \includegraphics[width=.9\linewidth]{figures/home-SPB.png} + \caption{The new SPB Portal.} + \label{fig:spb} +\end{figure} + diff --git a/OSS-2017/02-platform.tex b/OSS-2017/02-platform.tex index 4bdf7e0..c9bf605 100644 --- a/OSS-2017/02-platform.tex +++ b/OSS-2017/02-platform.tex @@ -1,60 +1,64 @@ -\section{Platform for software collaborative development} +\section{Architecture} -First, according to the Brazilian Federal Government requirements, we needed an -application that integrates existing OSS systems and provides a similar -interface for each one, as well as, unique authentication. For that, we have a -web-based integration platform called Colab that also works as a reverse proxy. -We have developed its plug-in architecture to decouple already integrated -tools (Trac system, GNU Mailman, and Apache Lucene Solr) and integrate other -platforms such as Noosfero, GitLab, and Mezuro. Figure \ref{fig:spb} shows the -home page of this integrated platform. +The two main requirements provided by the Brazilian Federal Government +for the new platform were: +% +1) \textit{Integrate existing FOSS systems}, with minimal differences +from their original versions. This way, the platform can benefit from +improvements done by the upstream communities that provide those +systems, and the maintenance effort that is specific for the SPB Portal +should be reduced; +% +and +2) \textit{Provide a consistent user interface} across the different +systems, as well as centralized authentication. -\begin{figure}[hbt] - \centering - \includegraphics[width=.9\linewidth]{figures/home-SPB.png} - \caption{The new SPB Portal.} - \label{fig:spb} -\end{figure} +The first requirement was accomplished by dedicating specialized teams +for each system that was being integrated. The teams would learn how to +develop their assigned systems, and contribute the necessary features +directly to the original communities, so that the version we used was +not significantly different from the original. Of course, at times +project deadlines forced us to use our own version before tho features +were fully reviewed and integrated upstream to the original projects, +but we managed to contribute the vast majority of the changes back. + +For the second requirement, we integrated a web integration platform +called Colab\footnote{\url{https://github.com/colab/colab}}. Colab +serves as a frontend for other web applications as a reverse proxy, +manages authentication, and can apply changes to the HTML provided by +the integrated applications in order to provide visual consistency. +Colab had support for an initial set of applications (Trac, GNU Mailman, +Apache Lucene) hard-coded; our team evolved Colab so that it can now +receive plugins to add support for new applications with minimal changes +to its existing core. We added support for the other applications used +in the SPB platform: Noosfero, GitLab, and Mezuro. -Noosfero is a software to build social and collaboration networks, providing a -platform with blogs and CMS (Content Management System). It is the environment -with most SPB user interaction providing features since the user registration -until the project pages with documentation and contacts. GitLab is a web-based -Git repository manager with wiki pages and issue tracking features to support -the SPB software collaborative development. Mezuro is a platform to collect -source code metric to monitor the internal quality of softwares written in C, -C++, Java, Python, Ruby, and PHP. +Noosfero\footnote{\url{http://noosfero.org/}} is a software for building +social and collaboration networks. Besides tha classical social +networking features, it also provides publication features such as blogs +and a general-purpose CMS (Content Management System). Most of the user +interactions with SPB is through Noosfero: user registration, project +home pages and documentation, and contact forms. +GitLab\footnote{\url{http://gitlab.com/}} is a web-based Git repository +manager with wiki pages and issue tracking features. +Mezuro\footnote{\url{http://mezuro.org/}} is a platform to collect source code +metric to monitor the internal quality of softwares written in C, C++, +Java, Python, Ruby, and PHP. GNU Mailman is used for mailing lists. + +The conceptual architecture of the platform is presented in Figure +\ref{fig:architecture}. Colab initially handles all user interaction, +directing requests to one of the integrated applications. It +post-processes responses from the applications to apply a consistent +visual appearance, manages authentication, and provides a unified search +functionality: instead of using the redundant restricted search +functionality of each application, a search in the SPB portal might +return content from any of the applications, be it web pages, mailing +list posts, or source code. \begin{figure}[hbt] \centering - \includegraphics[width=.8\linewidth]{figures/arquitetura.png} + \includegraphics[width=.6\linewidth]{figures/arch.png} \caption{SPB Portal Architecture.} \label{fig:architecture} \end{figure} -The new SPB Portal was deployed in 7 virtual machines with different functions, -as we can see in Figure \ref{fig:architecture}. The \textit{reverseproxy} -handles the HTTP requests and redirects them to the \textit{integration}, the -\textit{email} sends and receives e-mails on behalf of the platform and the -\textit{monitor} keeps the entire environment tracked. These three -\textit{VMs} mentioned - \textit{reverseproxy}, \textit{email} and -\textit{monitor} - are accessible via Internet and the other ones are only -available in the local network created between them. - -\textit{Integration} works as a second layer of proxy beneath -\textit{reverseproxy}, any request to the platform will be handled by it. The -Colab service provides interface, authentication and search engine integration -among all the services. When a request is received to a specific service, Colab -authenticates the user in the target tool, sends the request and makes a visual -transformation in the HTML page, which is the content of the response. Another -user-oriented feature is the integrated search engine, when the user want to -find something in the platform Colab will perform the search in the whole -databases. Colab itself provides a web interface for GNU Mailman and we have -two others integrated tools in \textit{integration}: Gitlab and Prezento (the -front-end of Mezuro). - -The source code static analysis is performed by \textit{mezuro}. It runs some -static analysis tools on source code stored in repository and provides this -data to Prezento. A social networking and CMS is provided by Noosfero in -\textit{social}, and the databases of all tools with a cache service are in -\textit{database}. diff --git a/OSS-2017/03-ux.tex b/OSS-2017/03-ux.tex index 920241e..3dc36da 100644 --- a/OSS-2017/03-ux.tex +++ b/OSS-2017/03-ux.tex @@ -3,34 +3,31 @@ The integration of collaborative environments goes beyond functional aspects. Offering the population an unified experience across these environments has been the key to encourage the use of the platform as it reduces the perception -of the complexity. Thus, the SPB Portal information architecture was redesigned +of complexity. Thus, the SPB Portal information architecture was redesigned to provide a transparent navigation and to reach users with different profiles. A process of harmonization has been employed on the interaction models of each -tool to reduce the learning curve. At the same time, a new visual style was +tool to reduce the learning curve. At the same time, a new visual style was created to unify the navigation experience and to comply with the guidelines of -the digital communication identity pattern established by the Federal +the digital communication identity standard established by the Federal Government. -With the increase of the system features and the addition of new tools, the +With the increase in system features and the addition of new tools, the visual style has steadily evolved to keep the navigation unified. Moreover, -tools from different backgrounds, which in many cases run functionalities with -similar concepts, made us face the challenge of developing a transparent -interface that unifies various context. For each required feature, we often -have to conciliate data with distinct structure and information pattern from -each used tool. The interface was the meeting point for this data with links -that users will not notice. The most common features (i.e. search, display, -and edition of profile information and content) have been brought in line with -the structure and other existing features of the SPB Portal. +tools from different backgrounds, which in many cases provide similar +functionality, prompted the development of an unified interface. Some +features, such as search and user profile editing were eliminated from +the individual applications, and implemented centrally to ensure a +consistent look and feel. -The other challenge is related to responsive web design. The SPB Portal -interface and each of its pages were designed following the bounds of -responsiveness provided or supported by each tools. In particular, in the case -of Noosfero, which do not have any official branch dedicated to improve its -responsiveness, we needed to engage with the expansion of this platform, where -we made a lot of contributions and dialogues with the community and -maintainers. +Another challenge was responsive web design. The integrated applications +had varying degrees of support for responsiveness, and the common +interface had to adapt for each individual scenario. In particular +Noosfero did not yet have a responsive design; we engaged in its +development and contributed towards that goal. -In summary, after the release of the new SPB Portal in 2014, several -validations activities were implemented in 2015 and 2016. The aim was to bring -the users closer to features that they have more interest and designing a -navigation which deepens in accordance with the user's depth of knowledge. +After the initial release of the new SPB Portal in 2014, several +validations activities were implemented in 2015 and 2016. The aim was to +provide the most wanted features by casual users (such as public +servants interested in downloads and documentation) immediately, while +allowing more experienced users (such as developers) to easily drill down +to the details. diff --git a/OSS-2017/04-finals.tex b/OSS-2017/04-finals.tex index 94fa2f8..a4ceeb4 100644 --- a/OSS-2017/04-finals.tex +++ b/OSS-2017/04-finals.tex @@ -1,17 +1,13 @@ \section{Final remarks} -The new SPB Portal was launched for approval in December 2014 and it is -available at \url{softwarepublico.gov.br}. All the integrated tools are OSS and -our contributions are published in open repositories, available on the SPB -Portal itself. We also returned these features to the respective communities. -Thus, it has reduced the costs of maintenance, which allowed us to carry out -the continued evolution of the platform in synergy with other organizations -that use the same tools. - -Providing a set of tools and improving the user experience in the portal is a -technical part of the SPB evolution process. Cultural and legal aspects of -network collaboration for an effective use of the new SPB Portal also was -matured. A study for licensing proposals and their impacts on the SPB was -conducted as a complement to what has been developed from the technological -point of view, generating a new legal instrument (\textit{Portaria} 46/2016 -STI/MP) on software collaborative development. +The portal is available at \url{softwarepublico.gov.br}. All +documentation, including detailed architecture and operation manuals are +also available\footnote{\url{https://softwarepublico.gov.br/doc/} +(in Portuguese only at the moment)}). +% +All the integrated tools are FOSS and our contributions were published +in open repositories, available on the SPB Portal itself. We also +contributed these features back to the respective communities: that +benefits those communities, as well as us since we can share future +development and maintenance effort with other organizations that +participate in their projects. diff --git a/OSS-2017/figures/arch.png b/OSS-2017/figures/arch.png new file mode 100644 index 0000000..833aea0 Binary files /dev/null and b/OSS-2017/figures/arch.png differ diff --git a/OSS-2017/figures/arch.svg b/OSS-2017/figures/arch.svg new file mode 100644 index 0000000..602d5e9 --- /dev/null +++ b/OSS-2017/figures/arch.svg @@ -0,0 +1,302 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + + Noosfero + + + + Colab + + + + Gitlab + + + + Mezuro + + + + Mailman + + + + internet + + + + + + + + + -- libgit2 0.21.2