Download as
Browse Code »

Commit c54bac8b997486ecbca298c90273f11b7717120b

Authored by Paulo Meireles
1 parent e470b925
Exists in master and in 3 other branches oss_2018, oss_2018_siqueira, rework_case_study

[opensym] reviewing the 'features' section

Showing 2 changed files with 207 additions and 196 deletions   Show diff stats
opensym2017/content/07-features.tex
  Diff comments   View file @c54bac8
1 1 \section{Features}
2 2 \label{sec:features}
3 3  
4   -The new generation of the SPB portal combines features adapted from existing collaborative software
5   -and features developed by the SPB team. Whenever possible, new functions
6   -(newly developed or partially modified) were contributed back to the official repositories.
7   -
8   -As a result, we have a platform that integrates and harmonizes different features such as
9   -social networking, mailing list, version control system, content management and
10   -source code quality monitoring. Our aim was to develop functionalities by reusing
11   -functionality of collaborative software already integrated to the platform. In
12   -addition, we tried to keep this integration transparent to end users.
13   -
14   -\subsection{Software and Software Community}
15   -
16   -In the new SPB portal, each software has a standard set of pages and tools.
17   -Besides accessing support pages (such as FAQ and installation guide) within the platform, users will be able to download
18   -different versions of the software and find several mechanisms of software development management.
19   -
20   -Focusing on the collaborative development, Mailman was integrated to the platform in order to allow
21   -the dialogue and communication between developers, users and
22   -enthusiasts of a determined software. The software has its own mailing list whose privacy
23   -can be configured/set by administrators.
24   -
25   -The software has a social interface area ("software community") where users can find other users, blogs,
26   -summary of recent activities, or any other relevant community-produced content.
27   -Users logged to the platform can request membership to different software communities
28   -and each community member can access and edit restricted content. For this purpose,
29   -many Noosfero features related to social networking and content management were integrated to the portal.
30   -
31   -To assist decision-making, the new SPB has acquired assessment and statistical
32   -tools. Now, users will be able to rate the software and make comments and all
33   -information will be avaiable to other users. Moreover, the software has a section
34   -containing its statistical data, where values are calculated based on data
35   -provided by users and the system.
36   -
37   -The role of the administrator will be present in the software and in its community. The
38   -administrator is responsible for moderating content, memberships and user
39   -comments. The administrator is also the one who can make changes in the software homepage
40   -content.
41   -
42   -\subsection{Software Catalog and global search}
43   -
44   -The platform also provides a search tool called Software Catalog,
45   -which allows users to find softwares available in the portal.
46   -In this catalog, some search options were developed to make the navigation easier,
47   -such as filters (by type of software or category), sorting and score.
48   -
49   -In order to expand the searching scope and cover more types of content, the SPB team
50   -developed the global search tool. This tool unifies search mechanisms
51   -provided by the different collaborative software used on the SPB portal. Any user can
52   -find a public content in the context of social networking, mailing list and
53   -software development.
54   -
55   -\subsection{Software development tools}
56   -
  4 +The new generation of the SPB Portal combines features adapted from existing
  5 +collaborative software and features developed by us. The new functions (newly
  6 +developed or partially modified) were contributed back to the official
  7 +repositories.
  8 +
  9 +As a result, we have a platform that integrates and harmonizes different
  10 +features such as social networking, mailing list, version control system,
  11 +content management, and source code quality monitoring. Our aim was to develop
  12 +functionalities by reusing functionality of collaborative software already
  13 +integrated to the platform. In addition, we tried to keep this integration
  14 +transparent to end users. We can discuss on the main features according to the
  15 +three groups below.
  16 +
  17 +\textbf{Software Project and Software Community.}
  18 +%
  19 +In the new SPB Portal, each software has a standard set of pages and tools.
  20 +Besides accessing support pages (such as FAQ and installation guide) within the
  21 +platform, users will be able to download different versions of the software and
  22 +find several mechanisms of software development management.
  23 +
  24 +Focusing on the collaborative development, Mailman was integrated to the
  25 +platform to allow the dialogue and communication between developers, users and
  26 +enthusiasts of a determined software. The software has its own mailing list
  27 +whose privacy can be configured by the software community administrators.
  28 +
  29 +The software has a social interface area (``software community'') where users
  30 +can find other users, blogs, summary of recent activities, or any other
  31 +relevant community-produced content. Users logged to the platform can request
  32 +membership to different software communities and each community member can
  33 +access and edit restricted content. For this purpose, many Noosfero features
  34 +related to social networking and content management were integrated to the
  35 +portal.
  36 +
  37 +To assist decision-making, the new SPB Portal has acquired assessment and
  38 +statistical tools. Now, users will be able to rate the software and make
  39 +comments and all information will be avaiable to other users. Moreover, the
  40 +software has a section containing its statistical data, where values are
  41 +calculated based on data provided by users and the system.
  42 +
  43 +The role of the administrator is present in the software project and in its
  44 +community. The administrator is responsible for moderating content, memberships
  45 +and user comments. The administrator is also the one who can make changes in
  46 +the software homepage content.
  47 +
  48 +\textbf{Software Catalog and Global Search.}
  49 +%
  50 +The platform also provides a search tool called Software Catalog, which allows
  51 +users to find softwares available in the portal. In this catalog, we developed
  52 +some search options to make the navigation easier, such as filters (by type of
  53 +software or category), sorting and score.
  54 +
  55 +To expand the searching scope and cover more types of content, we developed the
  56 +global search tool. This tool unifies search mechanisms provided by the
  57 +different collaborative software used on the SPB Portal. Any user can find a
  58 +public content in the context of social networking, mailing list, and software
  59 +repository.
  60 +
  61 +\textbf{Software Development Tools.}
  62 +%
57 63 Usually, Collaborative Development Environments (CDE) demand a version control
58 64 system, trackers, build tools, knowledge centers, and communication tools
59   -\cite{collaboration_tools}.
60   -The new SPB also provides
61   -tools to encourage developers to keep source code and its
62   -development activity within the platform. Any created software has, by default, a
63   -an associated git repository with wiki pages and issue tracking. These tools are
64   -supplied by the integration of Gitlab into the platform.
  65 +\cite{collaboration_tools}. The new SPB Portal also provides tools to encourage
  66 +developers to keep the source code and its development activity within the
  67 +platform. Any created software has, by default, an associated Git repository
  68 +with Wiki pages and issue tracking. These tools are supplied by the integration
  69 +of Gitlab into the platform.
65 70  
66 71 Developers can also evaluate the software source code to measure software
67   -quality. With Mezuro, they can schedule the analysis of the source-code and follow its
68   -metric results evolution over time. Results of each metric analysis are
69   -public, which allows greater transparency between the developer and the
  72 +quality. With Mezuro, they can schedule the analysis of the source code and
  73 +follow its metric results evolution over time. Results of each metric analysis
  74 +are public, which allows greater transparency between the developer and the
70 75 community that uses the software. Thereby, the maintainers can decide if the
71 76 given solution meets the source code quality requirements.
72 77  
73   -Thus, the SPB became a platform to stimulate the openness of the source code;
74   -dialogue between users and the development team; and also
75   -maintenance and evolution of the software, which will provide more
76   -transparency in government investments.
  78 +Thus, the SPB Portal became a platform to stimulate the openness of the source
  79 +code; dialogue between users and the development team; and also maintenance and
  80 +evolution of the software, which provide more transparency in Government
  81 +investments regarding to software development.
77 82  
... ...
opensym2017/content/09-conclusion.tex
  Diff comments   View file @c54bac8
1 1 \section{Conclusion}
2 2 \label{sec:conclusion}
3 3  
4   -In this work, we presented and discussed issues experienced during a government-funded
5   -project, in partnership with the University of Brasilia and the University of
6   -São Paulo, to evolve the Brazilian Public Software Portal.
  4 +In this work, we presented and discussed issues experienced during a
  5 +government-funded project, in partnership with the University of Brasilia and
  6 +the University of São Paulo, to evolve the Brazilian Public Software Portal.
7 7 Its contributions are twofold. First, we present the strategy used to develop
8   -and to deliver an unprecedented platform to Brazilian government. Second,
9   -based on the results of the SPB Portal project, we point out that it is
10   -possible to mitigate conflicts experienced in the development environment
11   -and to conciliate governmental and academy cultures. To summarize our main contributions, we answered in this section the three open questions those guided this paper.
  8 +and to deliver an unprecedented platform to Brazilian government. Second, based
  9 +on the results of the SPB Portal project, we point out that it is possible to
  10 +mitigate conflicts experienced in the development environment and to conciliate
  11 +governmental and academy cultures. To summarize our main contributions, we
  12 +answered in this section the three open questions those guided this paper.
12 13  
13   -\textbf{Q1:} \textit{Which strategy could be used to integrate several existing
  14 +\textbf{Which strategy could be used to integrate several existing
14 15 FLOSS tools to promote a collaborative software development?}
15 16 %
16   -The SPB portal integrates more than 10 FLOSS tools and provides several features,
17   -such as social network, mailing list, version control, content management and
18   -source code quality monitoring. Concerned with the platform sustainability and
19   -maintainability, the aforementioned 10 FLOSS tools were integrated with minimum
20   -differences from their official versions and the new developed features were
21   -sent upstream to ensure an alignment between the portal systems and their
22   -respective official versions. In the integration process, the main softwares
23   -were identified, specific teams were formed to work with each one of them
24   -and each team was composed of students with different levels of skills and at
25   -least one senior professional.
26   -
27   -\textbf{Q2:} \textit{How to involve students in real-world projects interacting with
  17 +The SPB portal integrates more than 10 FLOSS tools and provides several
  18 +features, such as social network, mailing list, version control, content
  19 +management and source code quality monitoring. Concerned with the platform
  20 +sustainability and maintainability, the aforementioned 10 FLOSS tools were
  21 +integrated with minimum differences from their official versions and the new
  22 +developed features were sent upstream to ensure an alignment between the portal
  23 +systems and their respective official versions. In the integration process, the
  24 +main softwares were identified, specific teams were formed to work with each
  25 +one of them and each team was composed of students with different levels of
  26 +skills and at least one senior professional.
  27 +
  28 +\textbf{How to involve students in real-world projects interacting with
28 29 real customers?}
29 30 %
30 31 In terms of mitigating conflicts, we tried to show that, as long as the
31 32 university can provide a healthy and challenging environment to its students,
32 33 one may conciliate studies and professional training in universities.
33 34 %
34   -The students interacted with professionals of diverse fields of expertise, and they were
35   -able to participate in all levels of the software development process.
36   -This contributed to a great learning opportunity.
  35 +The students interacted with professionals of diverse fields of expertise, and
  36 +they were able to participate in all levels of the software development
  37 +process. This contributed to a great learning opportunity.
37 38 %
38 39 In our work process, based on open and collaborative software development
39 40 practices, students could negotiate their work schedule as well as count on IT
... ... @@ -47,12 +48,13 @@ After the end of the project, some students successfully
47 48 embraced opportunities in public and private sectors, within national borders
48 49 and abroad. Some other students went further and started their own companies.
49 50  
50   -\textbf{Q3:} \textit{How to introduce collaborative and agile
  51 +\textbf{How to introduce collaborative and agile
51 52 practices typical in FLOSS environments in the governmental development process?}
52   -With some adaptations, what we called the ``translation processes'', it is feasible
53   -to conciliate agile methodologies and FLOSS practices to develop software to
54   -governmental organizations with functional hierarchical structures that use
55   -traditional development paradigm.
  53 +%
  54 +With some adaptations, what we called the ``translation processes'', it is
  55 +feasible to conciliate agile methodologies and FLOSS practices to develop
  56 +software to governmental organizations with functional hierarchical structures
  57 +that use traditional development paradigm.
56 58 %
57 59 Aiming at reducing client questions about workconclusion, a DevOps front was
58 60 created to automate all deploy process and also to work in continuous
... ... @@ -62,115 +64,119 @@ focused on providing a friendly working environment as well as on showing to
62 64 governmental agents another way to interact with the FLOSS community and the
63 65 university.
64 66  
65   -From the answers of our initial open questions, we can also highlighted six lessons learned to make easier to share our experience during the development of the new SPB Portal.
  67 +From the answers of our initial open questions, we can also highlighted six
  68 +lessons learned to make easier to share our experience during the development
  69 +of the new SPB Portal.
66 70  
67   -\textbf{L1:} \textit{The participation of experienced professionals is crucial to
  71 +\textbf{The participation of experienced professionals is crucial to
68 72 the success of the project.}
69   -One important factor for the students was the composition of the teams
70   -with the participation of experienced professionals.
71   -%
72   -On the technical side, the senior developers and designers would handle
73   -the more difficult technical decisions, creating a work environment
74   -where the students could develop their skills in a didactic way without
75   -pressure.
76   -%
77   -On the management side, the active participation of professors -- who
78   -are, in the end, the ones responsible for the project -- is crucial to
79   -make sure students participation is conducted in a healthy way, and it is an
80   -instance of leading by example.
81   -
82   -\textbf{L2:} \textit{A balanced relationship between academia and industry.}
83   -The experience of the SPB project led UnB to develop a work style which
84   -proved to be appropriate for an educational environment that brings
85   -academia and industry together.
86   -%
87   -The highest priority from the university's point of view is the
88   -students. Considering this, the activities of the project were never
89   -prioritized to the detriment of classes and other pedagogical
90   -activities. In summary, we had students working at different times, part
91   -time, remotely or locally, always respecting their individual
92   -conditions, but doing the work in a collective, collaborative and open
93   -way.
94   -%
95   -And even under a potentially adverse environment, the project delivered
96   -the desired solution with success.
97   -
98   -\textbf{L3:} \textit{Managing different organizational cultures.}
99   -In the beginning of the project, the Brazilian Government stakeholders
100   -had certain expectations about the project development that, let's
101   -say, did not exactly match our work style based on agile and FLOSS practices.
102   -%
103   -We had to develop strategies that would support these different
104   -organizational cultures. Therefore, we have created ``translation processes'' between our team and the MP managers who managed the project on their side, assuming a traditional waterfall process.
105   -
106   -\textbf{L4:} \textit{Managing higher-level and lower-level goals separately.}
107   -During the initial phase of the project, the MP team has brought
108   -strategic discussions to technical/operational meetings that
109   -were supposed to be about practical technical decisions.
  73 +One important factor for the students was the composition of the teams with the
  74 +participation of experienced professionals.
  75 +%
  76 +On the technical side, the senior developers and designers would handle the
  77 +more difficult technical decisions, creating a work environment where the
  78 +students could develop their skills in a didactic way without pressure.
  79 +%
  80 +On the management side, the active participation of professors -- who are, in
  81 +the end, the ones responsible for the project -- is crucial to make sure
  82 +students participation is conducted in a healthy way, and it is an instance of
  83 +leading by example.
  84 +
  85 +\textbf{A balanced relationship between academia and industry.}
  86 +The experience of the SPB project led UnB to develop a work style which proved
  87 +to be appropriate for an educational environment that brings academia and
  88 +industry together.
  89 +%
  90 +The highest priority from the university's point of view is the students.
  91 +Considering this, the activities of the project were never prioritized to the
  92 +detriment of classes and other pedagogical activities. In summary, we had
  93 +students working at different times, part time, remotely or locally, always
  94 +respecting their individual conditions, but doing the work in a collective,
  95 +collaborative and open way.
  96 +%
  97 +And even under a potentially adverse environment, the project delivered the
  98 +desired solution with success.
  99 +
  100 +\textbf{Managing different organizational cultures.}
  101 +%
  102 +In the beginning of the project, the Brazilian Government stakeholders had
  103 +certain expectations about the project development that, let's say, did not
  104 +exactly match our work style based on agile and FLOSS practices.
  105 +%
  106 +We had to develop strategies that would support these different organizational
  107 +cultures. Therefore, we have created ``translation processes'' between our team
  108 +and the MP managers who managed the project on their side, assuming a
  109 +traditional waterfall process.
  110 +
  111 +\textbf{Managing higher-level and lower-level goals separately.}
  112 +%
  113 +During the initial phase of the project, the MP team has brought strategic
  114 +discussions to technical/operational meetings that were supposed to be about
  115 +practical technical decisions.
110 116 %
111 117 This produced a highly complex communication and management environment,
112   -overloading the professors because they were supposed
113   -for maintaining the MP strategy synchronized with the
114   -implementation plans of the development team. This was hard, especially because the
115   -aforementioned cultural mismatch. Mixing both concerns in the same
116   -discussions caused confusion on both sides.
117   -%
118   -From the middle of the project we were able to keep those
119   -concerns separated, what eased the work of everyone involved.
120   -
121   -\textbf{L5:} \textit{Living with ill-advised political decisions.}
122   -At the initial phases of the project, by political and personal
123   -motivation, the main stakeholders from the Brazilian government imposed
124   -the use of Colab to guide the development of the new SPB platform. Our
125   -team was totally against the idea because we already knew that Colab was
126   -a very experimental project and its adoption could dramatically increase
127   -the project complexity. Even though we provided technical reasons to
128   -not utilize Colab, the MP was adamant and we had to manage this problem. We did massive changes to
129   -Colab, and at the end of the project we have completely rewritten it to make
130   -it stable. It is important to notice that the MP compelled us to accept
131   -a technical decision based only on political interests, without considering
132   -all the resources that would be spent due to that decision. At the end
133   -of the project, we verified that Colab indeed consumed a vast amount of
134   -the budget and increased the project complexity. At the end of the
135   -project and after our analysis on the decision made by the MP, we
136   -understand that MP managers are capable of ignoring technical reasons
137   -in favor of political decisions.
138   -
139   -\textbf{L6:} \textit{Consider sustainability from the beginning.}
140   -In the process of deploying the SPB platform in the MP infrastructure we had
141   -to interact with the MP technicians. We did several workshops, training
142   -and a meticulous documentation describing all the required procedures to
143   -update the platform, however, we realized that the MP technicians
144   -constantly avoid the responsibility. After noticing the aforementioned
145   -situation, we organized a DevOps team that completely automated all
146   -the deployment procedure. We simplified all the platform deployment to a
147   -few steps: (1) initial configurations (just ssh configuration) and (2)
148   -the execution of simple commands to completely update the platform. By
149   -the end of the project, we observed that the MP technicians invariably
150   -still depended on our support to update the platform even with all the
151   -automation provided by us. We were sadly left with a feeling of
152   -uncertainty about the future of the platform after the project ended. In
153   -hindsight, we realize that the MP dedicated system analysts and
154   -managers to the project, but not operations technicians. The later
155   -should have been more involved with the process so they could at least be
156   -comfortable in managing the platform infrastructure.
157   -
158   -
159   -Ultimately, the SPB portal is in production and its full
160   -documentation, including detailed architecture and operation manuals are
161   -also available\footnote{\url{https://softwarepublico.gov.br/doc/}}.
162   -%
163   -All the integrated tools are FLOSS and our contributions were published
164   -in open repositories, available on the SPB Portal itself. We also
165   -contributed these features back to the respective communities, which
166   -benefits both those communities and us, since we can share future
167   -development and maintenance effort with other organizations that
168   -participate in these projects.
  118 +overloading the professors because they were supposed for maintaining the MP
  119 +strategy synchronized with the implementation plans of the development team.
  120 +This was hard, especially because the aforementioned cultural mismatch. Mixing
  121 +both concerns in the same discussions caused confusion on both sides.
  122 +%
  123 +From the middle of the project we were able to keep those concerns separated,
  124 +what eased the work of everyone involved.
  125 +
  126 +\textbf{Living with ill-advised political decisions.}
  127 +%
  128 +At the initial phases of the project, by political and personal motivation, the
  129 +main stakeholders from the Brazilian government imposed the use of Colab to
  130 +guide the development of the new SPB platform. Our team was totally against the
  131 +idea because we already knew that Colab was a very experimental project and its
  132 +adoption could dramatically increase the project complexity. Even though we
  133 +provided technical reasons to not utilize Colab, the MP was adamant and we had
  134 +to manage this problem. We did massive changes to Colab, and at the end of the
  135 +project we have completely rewritten it to make it stable. It is important to
  136 +notice that the MP compelled us to accept a technical decision based only on
  137 +political interests, without considering all the resources that would be spent
  138 +due to that decision. At the end of the project, we verified that Colab indeed
  139 +consumed a vast amount of the budget and increased the project complexity. At
  140 +the end of the project and after our analysis on the decision made by the MP,
  141 +we understand that MP managers are capable of ignoring technical reasons in
  142 +favor of political decisions.
  143 +
  144 +\textbf{Consider sustainability from the beginning.}
  145 +%
  146 +In the process of deploying the SPB platform in the MP infrastructure we had to
  147 +interact with the MP technicians. We did several workshops, training and a
  148 +meticulous documentation describing all the required procedures to update the
  149 +platform, however, we realized that the MP technicians constantly avoid the
  150 +responsibility. After noticing the aforementioned situation, we organized a
  151 +DevOps team that completely automated all the deployment procedure. We
  152 +simplified all the platform deployment to a few steps: (1) initial
  153 +configurations (just ssh configuration) and (2) the execution of simple
  154 +commands to completely update the platform. By the end of the project, we
  155 +observed that the MP technicians invariably still depended on our support to
  156 +update the platform even with all the automation provided by us. We were sadly
  157 +left with a feeling of uncertainty about the future of the platform after the
  158 +project ended. In hindsight, we realize that the MP dedicated system analysts
  159 +and managers to the project, but not operations technicians. The later should
  160 +have been more involved with the process so they could at least be comfortable
  161 +in managing the platform infrastructure.
  162 +
  163 +
  164 +Ultimately, the SPB portal is in
  165 +production\footnote{\url{https://softwarepublico.gov.br}} and its full
  166 +documentation, including detailed architecture and operation manuals, is also
  167 +available\footnote{\url{https://softwarepublico.gov.br/doc/}}.
  168 +%
  169 +All the integrated tools are FLOSS and our contributions were published in open
  170 +repositories, available on the SPB Portal itself. We also contributed these
  171 +features back to the respective communities, which benefits both those
  172 +communities and us, since we can share future development and maintenance
  173 +effort with other organizations that participate in these projects.
169 174  
170 175 Future work should use data produced by the project to validate and evaluate
171 176 how the used FLOSS and Agile practices have impacted the students and the
172   -governmental development process. For this, we will conduce a \textit{postmortem}
173   -analysis using the project open data and a survey targeting the involved stakeholders.
  177 +governmental development process. For this, we will conduce a
  178 +\textit{postmortem} analysis using the project open data and a survey targeting
  179 +the involved stakeholders.
174 180  
175 181 %===========
176 182 % Conclusion
... ...