From c54bac8b997486ecbca298c90273f11b7717120b Mon Sep 17 00:00:00 2001 From: Paulo Meirelles Date: Tue, 25 Jul 2017 17:12:27 -0300 Subject: [PATCH] [opensym] reviewing the 'features' section --- opensym2017/content/07-features.tex | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------------------------ opensym2017/content/09-conclusion.tex | 266 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------------------------------------------------------------------------------------- 2 files changed, 207 insertions(+), 196 deletions(-) diff --git a/opensym2017/content/07-features.tex b/opensym2017/content/07-features.tex index e671eb7..003cbf5 100644 --- a/opensym2017/content/07-features.tex +++ b/opensym2017/content/07-features.tex @@ -1,77 +1,82 @@ \section{Features} \label{sec:features} -The new generation of the SPB portal combines features adapted from existing collaborative software -and features developed by the SPB team. Whenever possible, new functions -(newly developed or partially modified) were contributed back to the official repositories. - -As a result, we have a platform that integrates and harmonizes different features such as -social networking, mailing list, version control system, content management and -source code quality monitoring. Our aim was to develop functionalities by reusing -functionality of collaborative software already integrated to the platform. In -addition, we tried to keep this integration transparent to end users. - -\subsection{Software and Software Community} - -In the new SPB portal, each software has a standard set of pages and tools. -Besides accessing support pages (such as FAQ and installation guide) within the platform, users will be able to download -different versions of the software and find several mechanisms of software development management. - -Focusing on the collaborative development, Mailman was integrated to the platform in order to allow -the dialogue and communication between developers, users and -enthusiasts of a determined software. The software has its own mailing list whose privacy -can be configured/set by administrators. - -The software has a social interface area ("software community") where users can find other users, blogs, -summary of recent activities, or any other relevant community-produced content. -Users logged to the platform can request membership to different software communities -and each community member can access and edit restricted content. For this purpose, -many Noosfero features related to social networking and content management were integrated to the portal. - -To assist decision-making, the new SPB has acquired assessment and statistical -tools. Now, users will be able to rate the software and make comments and all -information will be avaiable to other users. Moreover, the software has a section -containing its statistical data, where values are calculated based on data -provided by users and the system. - -The role of the administrator will be present in the software and in its community. The -administrator is responsible for moderating content, memberships and user -comments. The administrator is also the one who can make changes in the software homepage -content. - -\subsection{Software Catalog and global search} - -The platform also provides a search tool called Software Catalog, -which allows users to find softwares available in the portal. -In this catalog, some search options were developed to make the navigation easier, -such as filters (by type of software or category), sorting and score. - -In order to expand the searching scope and cover more types of content, the SPB team -developed the global search tool. This tool unifies search mechanisms -provided by the different collaborative software used on the SPB portal. Any user can -find a public content in the context of social networking, mailing list and -software development. - -\subsection{Software development tools} - +The new generation of the SPB Portal combines features adapted from existing +collaborative software and features developed by us. The new functions (newly +developed or partially modified) were contributed back to the official +repositories. + +As a result, we have a platform that integrates and harmonizes different +features such as social networking, mailing list, version control system, +content management, and source code quality monitoring. Our aim was to develop +functionalities by reusing functionality of collaborative software already +integrated to the platform. In addition, we tried to keep this integration +transparent to end users. We can discuss on the main features according to the +three groups below. + +\textbf{Software Project and Software Community.} +% +In the new SPB Portal, each software has a standard set of pages and tools. +Besides accessing support pages (such as FAQ and installation guide) within the +platform, users will be able to download different versions of the software and +find several mechanisms of software development management. + +Focusing on the collaborative development, Mailman was integrated to the +platform to allow the dialogue and communication between developers, users and +enthusiasts of a determined software. The software has its own mailing list +whose privacy can be configured by the software community administrators. + +The software has a social interface area (``software community'') where users +can find other users, blogs, summary of recent activities, or any other +relevant community-produced content. Users logged to the platform can request +membership to different software communities and each community member can +access and edit restricted content. For this purpose, many Noosfero features +related to social networking and content management were integrated to the +portal. + +To assist decision-making, the new SPB Portal has acquired assessment and +statistical tools. Now, users will be able to rate the software and make +comments and all information will be avaiable to other users. Moreover, the +software has a section containing its statistical data, where values are +calculated based on data provided by users and the system. + +The role of the administrator is present in the software project and in its +community. The administrator is responsible for moderating content, memberships +and user comments. The administrator is also the one who can make changes in +the software homepage content. + +\textbf{Software Catalog and Global Search.} +% +The platform also provides a search tool called Software Catalog, which allows +users to find softwares available in the portal. In this catalog, we developed +some search options to make the navigation easier, such as filters (by type of +software or category), sorting and score. + +To expand the searching scope and cover more types of content, we developed the +global search tool. This tool unifies search mechanisms provided by the +different collaborative software used on the SPB Portal. Any user can find a +public content in the context of social networking, mailing list, and software +repository. + +\textbf{Software Development Tools.} +% Usually, Collaborative Development Environments (CDE) demand a version control system, trackers, build tools, knowledge centers, and communication tools -\cite{collaboration_tools}. -The new SPB also provides -tools to encourage developers to keep source code and its -development activity within the platform. Any created software has, by default, a -an associated git repository with wiki pages and issue tracking. These tools are -supplied by the integration of Gitlab into the platform. +\cite{collaboration_tools}. The new SPB Portal also provides tools to encourage +developers to keep the source code and its development activity within the +platform. Any created software has, by default, an associated Git repository +with Wiki pages and issue tracking. These tools are supplied by the integration +of Gitlab into the platform. Developers can also evaluate the software source code to measure software -quality. With Mezuro, they can schedule the analysis of the source-code and follow its -metric results evolution over time. Results of each metric analysis are -public, which allows greater transparency between the developer and the +quality. With Mezuro, they can schedule the analysis of the source code and +follow its metric results evolution over time. Results of each metric analysis +are public, which allows greater transparency between the developer and the community that uses the software. Thereby, the maintainers can decide if the given solution meets the source code quality requirements. -Thus, the SPB became a platform to stimulate the openness of the source code; -dialogue between users and the development team; and also -maintenance and evolution of the software, which will provide more -transparency in government investments. +Thus, the SPB Portal became a platform to stimulate the openness of the source +code; dialogue between users and the development team; and also maintenance and +evolution of the software, which provide more transparency in Government +investments regarding to software development. diff --git a/opensym2017/content/09-conclusion.tex b/opensym2017/content/09-conclusion.tex index ce2f545..b295979 100644 --- a/opensym2017/content/09-conclusion.tex +++ b/opensym2017/content/09-conclusion.tex @@ -1,39 +1,40 @@ \section{Conclusion} \label{sec:conclusion} -In this work, we presented and discussed issues experienced during a government-funded -project, in partnership with the University of Brasilia and the University of -São Paulo, to evolve the Brazilian Public Software Portal. +In this work, we presented and discussed issues experienced during a +government-funded project, in partnership with the University of Brasilia and +the University of São Paulo, to evolve the Brazilian Public Software Portal. Its contributions are twofold. First, we present the strategy used to develop -and to deliver an unprecedented platform to Brazilian government. Second, -based on the results of the SPB Portal project, we point out that it is -possible to mitigate conflicts experienced in the development environment -and to conciliate governmental and academy cultures. To summarize our main contributions, we answered in this section the three open questions those guided this paper. +and to deliver an unprecedented platform to Brazilian government. Second, based +on the results of the SPB Portal project, we point out that it is possible to +mitigate conflicts experienced in the development environment and to conciliate +governmental and academy cultures. To summarize our main contributions, we +answered in this section the three open questions those guided this paper. -\textbf{Q1:} \textit{Which strategy could be used to integrate several existing +\textbf{Which strategy could be used to integrate several existing FLOSS tools to promote a collaborative software development?} % -The SPB portal integrates more than 10 FLOSS tools and provides several features, -such as social network, mailing list, version control, content management and -source code quality monitoring. Concerned with the platform sustainability and -maintainability, the aforementioned 10 FLOSS tools were integrated with minimum -differences from their official versions and the new developed features were -sent upstream to ensure an alignment between the portal systems and their -respective official versions. In the integration process, the main softwares -were identified, specific teams were formed to work with each one of them -and each team was composed of students with different levels of skills and at -least one senior professional. - -\textbf{Q2:} \textit{How to involve students in real-world projects interacting with +The SPB portal integrates more than 10 FLOSS tools and provides several +features, such as social network, mailing list, version control, content +management and source code quality monitoring. Concerned with the platform +sustainability and maintainability, the aforementioned 10 FLOSS tools were +integrated with minimum differences from their official versions and the new +developed features were sent upstream to ensure an alignment between the portal +systems and their respective official versions. In the integration process, the +main softwares were identified, specific teams were formed to work with each +one of them and each team was composed of students with different levels of +skills and at least one senior professional. + +\textbf{How to involve students in real-world projects interacting with real customers?} % In terms of mitigating conflicts, we tried to show that, as long as the university can provide a healthy and challenging environment to its students, one may conciliate studies and professional training in universities. % -The students interacted with professionals of diverse fields of expertise, and they were -able to participate in all levels of the software development process. -This contributed to a great learning opportunity. +The students interacted with professionals of diverse fields of expertise, and +they were able to participate in all levels of the software development +process. This contributed to a great learning opportunity. % In our work process, based on open and collaborative software development practices, students could negotiate their work schedule as well as count on IT @@ -47,12 +48,13 @@ After the end of the project, some students successfully embraced opportunities in public and private sectors, within national borders and abroad. Some other students went further and started their own companies. -\textbf{Q3:} \textit{How to introduce collaborative and agile +\textbf{How to introduce collaborative and agile practices typical in FLOSS environments in the governmental development process?} -With some adaptations, what we called the ``translation processes'', it is feasible -to conciliate agile methodologies and FLOSS practices to develop software to -governmental organizations with functional hierarchical structures that use -traditional development paradigm. +% +With some adaptations, what we called the ``translation processes'', it is +feasible to conciliate agile methodologies and FLOSS practices to develop +software to governmental organizations with functional hierarchical structures +that use traditional development paradigm. % Aiming at reducing client questions about workconclusion, a DevOps front was created to automate all deploy process and also to work in continuous @@ -62,115 +64,119 @@ focused on providing a friendly working environment as well as on showing to governmental agents another way to interact with the FLOSS community and the university. -From the answers of our initial open questions, we can also highlighted six lessons learned to make easier to share our experience during the development of the new SPB Portal. +From the answers of our initial open questions, we can also highlighted six +lessons learned to make easier to share our experience during the development +of the new SPB Portal. -\textbf{L1:} \textit{The participation of experienced professionals is crucial to +\textbf{The participation of experienced professionals is crucial to the success of the project.} -One important factor for the students was the composition of the teams -with the participation of experienced professionals. -% -On the technical side, the senior developers and designers would handle -the more difficult technical decisions, creating a work environment -where the students could develop their skills in a didactic way without -pressure. -% -On the management side, the active participation of professors -- who -are, in the end, the ones responsible for the project -- is crucial to -make sure students participation is conducted in a healthy way, and it is an -instance of leading by example. - -\textbf{L2:} \textit{A balanced relationship between academia and industry.} -The experience of the SPB project led UnB to develop a work style which -proved to be appropriate for an educational environment that brings -academia and industry together. -% -The highest priority from the university's point of view is the -students. Considering this, the activities of the project were never -prioritized to the detriment of classes and other pedagogical -activities. In summary, we had students working at different times, part -time, remotely or locally, always respecting their individual -conditions, but doing the work in a collective, collaborative and open -way. -% -And even under a potentially adverse environment, the project delivered -the desired solution with success. - -\textbf{L3:} \textit{Managing different organizational cultures.} -In the beginning of the project, the Brazilian Government stakeholders -had certain expectations about the project development that, let's -say, did not exactly match our work style based on agile and FLOSS practices. -% -We had to develop strategies that would support these different -organizational cultures. Therefore, we have created ``translation processes'' between our team and the MP managers who managed the project on their side, assuming a traditional waterfall process. - -\textbf{L4:} \textit{Managing higher-level and lower-level goals separately.} -During the initial phase of the project, the MP team has brought -strategic discussions to technical/operational meetings that -were supposed to be about practical technical decisions. +One important factor for the students was the composition of the teams with the +participation of experienced professionals. +% +On the technical side, the senior developers and designers would handle the +more difficult technical decisions, creating a work environment where the +students could develop their skills in a didactic way without pressure. +% +On the management side, the active participation of professors -- who are, in +the end, the ones responsible for the project -- is crucial to make sure +students participation is conducted in a healthy way, and it is an instance of +leading by example. + +\textbf{A balanced relationship between academia and industry.} +The experience of the SPB project led UnB to develop a work style which proved +to be appropriate for an educational environment that brings academia and +industry together. +% +The highest priority from the university's point of view is the students. +Considering this, the activities of the project were never prioritized to the +detriment of classes and other pedagogical activities. In summary, we had +students working at different times, part time, remotely or locally, always +respecting their individual conditions, but doing the work in a collective, +collaborative and open way. +% +And even under a potentially adverse environment, the project delivered the +desired solution with success. + +\textbf{Managing different organizational cultures.} +% +In the beginning of the project, the Brazilian Government stakeholders had +certain expectations about the project development that, let's say, did not +exactly match our work style based on agile and FLOSS practices. +% +We had to develop strategies that would support these different organizational +cultures. Therefore, we have created ``translation processes'' between our team +and the MP managers who managed the project on their side, assuming a +traditional waterfall process. + +\textbf{Managing higher-level and lower-level goals separately.} +% +During the initial phase of the project, the MP team has brought strategic +discussions to technical/operational meetings that were supposed to be about +practical technical decisions. % This produced a highly complex communication and management environment, -overloading the professors because they were supposed -for maintaining the MP strategy synchronized with the -implementation plans of the development team. This was hard, especially because the -aforementioned cultural mismatch. Mixing both concerns in the same -discussions caused confusion on both sides. -% -From the middle of the project we were able to keep those -concerns separated, what eased the work of everyone involved. - -\textbf{L5:} \textit{Living with ill-advised political decisions.} -At the initial phases of the project, by political and personal -motivation, the main stakeholders from the Brazilian government imposed -the use of Colab to guide the development of the new SPB platform. Our -team was totally against the idea because we already knew that Colab was -a very experimental project and its adoption could dramatically increase -the project complexity. Even though we provided technical reasons to -not utilize Colab, the MP was adamant and we had to manage this problem. We did massive changes to -Colab, and at the end of the project we have completely rewritten it to make -it stable. It is important to notice that the MP compelled us to accept -a technical decision based only on political interests, without considering -all the resources that would be spent due to that decision. At the end -of the project, we verified that Colab indeed consumed a vast amount of -the budget and increased the project complexity. At the end of the -project and after our analysis on the decision made by the MP, we -understand that MP managers are capable of ignoring technical reasons -in favor of political decisions. - -\textbf{L6:} \textit{Consider sustainability from the beginning.} -In the process of deploying the SPB platform in the MP infrastructure we had -to interact with the MP technicians. We did several workshops, training -and a meticulous documentation describing all the required procedures to -update the platform, however, we realized that the MP technicians -constantly avoid the responsibility. After noticing the aforementioned -situation, we organized a DevOps team that completely automated all -the deployment procedure. We simplified all the platform deployment to a -few steps: (1) initial configurations (just ssh configuration) and (2) -the execution of simple commands to completely update the platform. By -the end of the project, we observed that the MP technicians invariably -still depended on our support to update the platform even with all the -automation provided by us. We were sadly left with a feeling of -uncertainty about the future of the platform after the project ended. In -hindsight, we realize that the MP dedicated system analysts and -managers to the project, but not operations technicians. The later -should have been more involved with the process so they could at least be -comfortable in managing the platform infrastructure. - - -Ultimately, the SPB portal is in production and its full -documentation, including detailed architecture and operation manuals are -also available\footnote{\url{https://softwarepublico.gov.br/doc/}}. -% -All the integrated tools are FLOSS and our contributions were published -in open repositories, available on the SPB Portal itself. We also -contributed these features back to the respective communities, which -benefits both those communities and us, since we can share future -development and maintenance effort with other organizations that -participate in these projects. +overloading the professors because they were supposed for maintaining the MP +strategy synchronized with the implementation plans of the development team. +This was hard, especially because the aforementioned cultural mismatch. Mixing +both concerns in the same discussions caused confusion on both sides. +% +From the middle of the project we were able to keep those concerns separated, +what eased the work of everyone involved. + +\textbf{Living with ill-advised political decisions.} +% +At the initial phases of the project, by political and personal motivation, the +main stakeholders from the Brazilian government imposed the use of Colab to +guide the development of the new SPB platform. Our team was totally against the +idea because we already knew that Colab was a very experimental project and its +adoption could dramatically increase the project complexity. Even though we +provided technical reasons to not utilize Colab, the MP was adamant and we had +to manage this problem. We did massive changes to Colab, and at the end of the +project we have completely rewritten it to make it stable. It is important to +notice that the MP compelled us to accept a technical decision based only on +political interests, without considering all the resources that would be spent +due to that decision. At the end of the project, we verified that Colab indeed +consumed a vast amount of the budget and increased the project complexity. At +the end of the project and after our analysis on the decision made by the MP, +we understand that MP managers are capable of ignoring technical reasons in +favor of political decisions. + +\textbf{Consider sustainability from the beginning.} +% +In the process of deploying the SPB platform in the MP infrastructure we had to +interact with the MP technicians. We did several workshops, training and a +meticulous documentation describing all the required procedures to update the +platform, however, we realized that the MP technicians constantly avoid the +responsibility. After noticing the aforementioned situation, we organized a +DevOps team that completely automated all the deployment procedure. We +simplified all the platform deployment to a few steps: (1) initial +configurations (just ssh configuration) and (2) the execution of simple +commands to completely update the platform. By the end of the project, we +observed that the MP technicians invariably still depended on our support to +update the platform even with all the automation provided by us. We were sadly +left with a feeling of uncertainty about the future of the platform after the +project ended. In hindsight, we realize that the MP dedicated system analysts +and managers to the project, but not operations technicians. The later should +have been more involved with the process so they could at least be comfortable +in managing the platform infrastructure. + + +Ultimately, the SPB portal is in +production\footnote{\url{https://softwarepublico.gov.br}} and its full +documentation, including detailed architecture and operation manuals, is also +available\footnote{\url{https://softwarepublico.gov.br/doc/}}. +% +All the integrated tools are FLOSS and our contributions were published in open +repositories, available on the SPB Portal itself. We also contributed these +features back to the respective communities, which benefits both those +communities and us, since we can share future development and maintenance +effort with other organizations that participate in these projects. Future work should use data produced by the project to validate and evaluate how the used FLOSS and Agile practices have impacted the students and the -governmental development process. For this, we will conduce a \textit{postmortem} -analysis using the project open data and a survey targeting the involved stakeholders. +governmental development process. For this, we will conduce a +\textit{postmortem} analysis using the project open data and a survey targeting +the involved stakeholders. %=========== % Conclusion -- libgit2 0.21.2