content/04-architecture.tex: review

Antonio Terceiro
1 parent bee1d5d6
Showing 1 changed file with 96 additions and 86 deletions Show diff stats
opensym2017/content/04-architecture.tex
 \section{Architecture}
 \label{sec:architecture}
-Based on the great list of functional requirements desired by Brazilian Federal
-Government we decided to select some FOSS systems that already contemplate some
-of them and improve these systems. And bringing the idea of community, it is
-undoable build a platform to be used by communities, which is a complex
-scenario, using just one tool.
+Based on the extensive list of functional requirements defined by the
+Brazilian Federal Government, we selected some FOSS systems to form our
+solution. We looked for system that together could provide the largest
+subset possible of the requirements, and were fully aware that we would
+need to improve those systems in order to provide the rest. We were also
+convinced that it would be impossible to provide all of those
+requirements with a single tool.
-At the point of view of the architecture, two main requirements was brought by
-the Brazilian Federal Government for the new platform:
+From the point of view of the architecture, two main requirements was
+brought by the Brazilian Federal Government for the new platform:
 \begin{enumerate}
   \item \textit{Integrate existing FOSS systems}, with minimal differences from
@@ -17,69 +19,81 @@ the Brazilian Federal Government for the new platform:
     systems, as well as centralized authentication.
 \end{enumerate}
-Adopting existing FOSS systems by the government could bring the benefit of
-improvements done by the upstream communities, and the maintenance effort. On
-the other hand, integrating different tools with distinct intent, it is not an
-easy task and it was important to have a consistent user interface which
-justifies the last requirement.
+Adopting existing FOSS systems and minimizing locally-made changes had
+the objective of being able to upgrade to newer versions of the original
+software, benefiting from improvements and maintenance done by the
+existing project communities. Providing a consistent user interface on
+top of those different tools was needed to make the transition between
+the different systems seamless from the point of view of users, which
+would be confused by jumping through completely different interfaces
+while interacting with the portal.
 For the first requirement, we identified four main systems that required
-specialized teams for work in the integration process. The teams learned how to
-develop for their assigned systems and contributed to the  original
-communities, so that the version we used were not significantly different from
-the original.
+specialized teams for work in the integration process. The teams learned
+how to develop for their assigned systems and contributed to the
+original communities, so that the version we used were not significantly
+different from the original.
-At the end of the project, SPB portal was composed of more than ten systems
-among them we can highlight: Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix,
-Munin, and so forth. The following sections explained a little bit of Colab,
-Noosfero, Mezuro, and Gitlab (the main tools which we contributed). Below, we
-described how we integrated those tools and conclude with the deployment.
+At the end of the project, the SPB portal was composed of more than ten
+systems, such as Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix, and
+Munin. The remainder of this section describes the most relevant of
+them, as well as how they were integrated into the platform.
 \subsection{Colab}
-Colab integrates web applications as its main goal, the user of this composed
-system should not notice the change between the integrated applications. Colab
-was designed to use one plugin for each system under its domain, this is
-guaranteed by four levels of integration:
+Colab\footnote{\url{https://github.com/colab}} is a systems integration
+platform for web applications. One of its goals is allowing different
+applications to be combined in such a way that an user does not notice
+the change between applications. For that, Colab provides facilities
+for:
 \begin{itemize}
-  \item Authentication
-  \item Visual
-  \item Events
-  \item Data and search engine
+  \item Centralized authentication
+  \item Visual consistency
+  \item Relaying of events between applications
+  \item Integrated search engine
 \end{itemize}
-The aforementioned integrations levels were possible, because Colab works as a
-reverse proxy, therefore all external requests pass through it.
-
-Single Sign-On (SSO) is used to login users throughout all integrated
-applications. REMOTE\_USER HTTP header is sent to the applications and we
-expect that they know how to handle it, since this is a common authentication
-mechanism. The integrated applications should be on a local network to avoid
-security issues. With this the user will be able to navigate through the
-platform applications and will not be asked for authentication credentials.
-
-As Colab\footnote{\url{https://github.com/colab}} is a reverse proxy, it makes
-some HTML transformations in the HTTP response of integrated applications to
-provide a unified interface. Allows one to define some default templates to be
-used by all applications and overwrite when needed in its plugin. This approach
-allowed us to reuse some HTML pages which facilitates maintenance.
-
-A publish-subscribe implementation was used to handle events in the platform.
-Thus, if some application want to trigger some action in case that other
-application do something is possible. A registration of the desired events and
-implementation of the handlers must be done in the plugin of each application.
-This bring us many options to innovate in these integrations.
-
-A integrated search engine is provided by Colab. Each plugin specify which data
-will be indexed and will became available for the users, just need an simple
-implementation of how should perform the collection of data.
+Colab implements this integration by working as a reverse proxy for the
+integration applications, that is, all external requests pass through
+Colab before reaching them.
+
+Centralized authentication is handled by letting users authenticate to
+Colab, which then sends a REMOTE\_USER HTTP header to applications,
+which are expected to be pre-configured to accept that as an indication
+of the current user (REMOTE\_USER is a standard authentication mechanism
+for web applications). This allows users to be automatically identified
+to each of the applications without having to enter credentials for each
+of them.
+%
+Of course, this requires that the integrated applications are not
+accessible on the public internet, otherwise malicious users could send
+their own crafted REMOTE\_USER headers and impersonate any user.
+
+For the visual consistency, Colab is able to make transformations to the
+HTML produced by the integrated applications, ir order to provide a
+unified interface. It allows one to define default templates to be used
+by all applications, as well as providing extra ones in a plugin. This
+approach allowed us to reuse common HTML pages, what facilitates
+maintenance.
+
+Colab also provides a publish-subscribe event system. This allows one of
+the applications, or Colab itself, to trigger an action in another
+application. For example, when a user registers with Colab, all
+applications need to be notified in order to create their own internal
+representation of that user account.
+
+Colab also provides an integrated search engine, which can be configured
+to specify exactly what data needs to be indexed for each of the
+applications, and how to direct the user to that piece of information on
+the specific application.
 Initially, Colab had support for a small set of applications (Trac, GNU
-Mailman, and Apache Lucene) and all of them was hard-coded. Our team evolved
-Colab so that it can now receive plugins to add support for new applications
-with minimal changes to its existing core. We developed plugins to be used in
-the SPB platform: Noosfero, GitLab, and Mezuro.
+Mailman, and Apache Lucene) and support for them was hard-coded. Our
+team evolved Colab so that it can now receive plugins that add support
+new applications without the need of changes to the Colab core. We also
+developed plugins to be used in the SPB platform: Noosfero, GitLab, and
+Mezuro.
 \subsection{Noosfero}
@@ -92,38 +106,34 @@ home pages and documentation, and contact forms.
 \subsection{Gitlab}
-GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository manager
-with wiki pages and issue tracking features. On the one hand, Github is the
-most famous and used Git repository manager. On the other hand, it is not a
-FOSS. Gitlab is a FOSS platform and focuses on delivering a holistic solution
-that will see developers from idea to production seamlessly and on a single
-platform. Thus, we have worked on it to integrated to Colab.
-
-Moreover, Gitlab has several features working properly than Github that is
-interesting in SPB context such as free for private projects, built-in
-continuous integration and continuous deployment with best practices, more
-control during downtime and upgrade, flexible permissions, Work-in-Progress
-protection, move issues between projects, Group-level milestones, Create new
-branches from issues, Issue board, Time tracking, as well new features and
-improvements every month on the 22nd.
+GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository
+manager with wiki pages and issue tracking features. Gitlab is a FOSS
+platform and focuses on delivering a holistic solution that will see
+developers from idea to production seamlessly and on a single platform.
+
+Gitlab has several unique features, such as built-in continuous
+integration and continuous deployment, flexible permissions, tracking of
+Work-in-Progress work, moving issues between projects, group-level
+milestones, creating new branches from issues, issues board, and time
+tracking.
 \subsection{Mezuro}
 Mezuro\footnote{\url{http://mezuro.org/}} is a platform to collect source code
-metric to monitor the internal quality of softwares written in C, C++,
-Java, Python, Ruby, and PHP. GNU Mailman is used for mailing lists.
-
-In general, source code metric tools also do not present a friendly way to
-interpret its results and, even more, do not follow a standardization between
-them. Mezuro brings and presents these results to
-the end user, specially, by analyzing source code metric history during its
-life cycle.
-
-The Mezuro platform provides a single interface
-grouping available tools, allows selection and composition of metrics in a
-flexible manner, stores the metrics evolution history, presents
-results in a friendly way, as well as, allows users to customize the given
-interpretation accordingly to their own context.
+metrics to monitor the internal quality of software written in C, C++,
+Java, Python, Ruby, and PHP.
+
+In general, source code metrics tools also do not present a friendly way
+to interpret its results and, even more, do not follow a standardization
+between them. Mezuro collects and presents these results to the end
+user, specially, by analyzing source code metric history during its life
+cycle.
+
+The Mezuro platform provides a single interface grouping available
+tools, allows selection and composition of metrics in a flexible manner,
+stores the metrics evolution history, presents results in a friendly
+way, as well as, allows users to customize the given interpretation
+accordingly to their own context.
 \subsection{System unification}
1	\section{Architecture}	1	\section{Architecture}
2	\label{sec:architecture}	2	\label{sec:architecture}
3		3
4	-Based on the great list of functional requirements desired by Brazilian Federal
5	-Government we decided to select some FOSS systems that already contemplate some
6	-of them and improve these systems. And bringing the idea of community, it is
7	-undoable build a platform to be used by communities, which is a complex
8	-scenario, using just one tool.	4	+Based on the extensive list of functional requirements defined by the
		5	+Brazilian Federal Government, we selected some FOSS systems to form our
		6	+solution. We looked for system that together could provide the largest
		7	+subset possible of the requirements, and were fully aware that we would
		8	+need to improve those systems in order to provide the rest. We were also
		9	+convinced that it would be impossible to provide all of those
		10	+requirements with a single tool.
9		11
10	-At the point of view of the architecture, two main requirements was brought by
11	-the Brazilian Federal Government for the new platform:	12	+From the point of view of the architecture, two main requirements was
		13	+brought by the Brazilian Federal Government for the new platform:
12		14
13	\begin{enumerate}	15	\begin{enumerate}
14	\item \textit{Integrate existing FOSS systems}, with minimal differences from	16	\item \textit{Integrate existing FOSS systems}, with minimal differences from
	@@ -17,69 +19,81 @@ the Brazilian Federal Government for the new platform:		@@ -17,69 +19,81 @@ the Brazilian Federal Government for the new platform:
17	systems, as well as centralized authentication.	19	systems, as well as centralized authentication.
18	\end{enumerate}	20	\end{enumerate}
19		21
20	-Adopting existing FOSS systems by the government could bring the benefit of
21	-improvements done by the upstream communities, and the maintenance effort. On
22	-the other hand, integrating different tools with distinct intent, it is not an
23	-easy task and it was important to have a consistent user interface which
24	-justifies the last requirement.	22	+Adopting existing FOSS systems and minimizing locally-made changes had
		23	+the objective of being able to upgrade to newer versions of the original
		24	+software, benefiting from improvements and maintenance done by the
		25	+existing project communities. Providing a consistent user interface on
		26	+top of those different tools was needed to make the transition between
		27	+the different systems seamless from the point of view of users, which
		28	+would be confused by jumping through completely different interfaces
		29	+while interacting with the portal.
25		30
26	For the first requirement, we identified four main systems that required	31	For the first requirement, we identified four main systems that required
27	-specialized teams for work in the integration process. The teams learned how to
28	-develop for their assigned systems and contributed to the original
29	-communities, so that the version we used were not significantly different from
30	-the original.	32	+specialized teams for work in the integration process. The teams learned
		33	+how to develop for their assigned systems and contributed to the
		34	+original communities, so that the version we used were not significantly
		35	+different from the original.
31		36
32	-At the end of the project, SPB portal was composed of more than ten systems
33	-among them we can highlight: Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix,
34	-Munin, and so forth. The following sections explained a little bit of Colab,
35	-Noosfero, Mezuro, and Gitlab (the main tools which we contributed). Below, we
36	-described how we integrated those tools and conclude with the deployment.	37	+At the end of the project, the SPB portal was composed of more than ten
		38	+systems, such as Colab, Noosfero, Mezuro, Gitlab, Mailman, Postfix, and
		39	+Munin. The remainder of this section describes the most relevant of
		40	+them, as well as how they were integrated into the platform.
37		41
38	\subsection{Colab}	42	\subsection{Colab}
39		43
40	-Colab integrates web applications as its main goal, the user of this composed
41	-system should not notice the change between the integrated applications. Colab
42	-was designed to use one plugin for each system under its domain, this is
43	-guaranteed by four levels of integration:	44	+Colab\footnote{\url{https://github.com/colab}} is a systems integration
		45	+platform for web applications. One of its goals is allowing different
		46	+applications to be combined in such a way that an user does not notice
		47	+the change between applications. For that, Colab provides facilities
		48	+for:
44		49
45	\begin{itemize}	50	\begin{itemize}
46	- \item Authentication
47	- \item Visual
48	- \item Events
49	- \item Data and search engine	51	+ \item Centralized authentication
		52	+ \item Visual consistency
		53	+ \item Relaying of events between applications
		54	+ \item Integrated search engine
50	\end{itemize}	55	\end{itemize}
51		56
52	-The aforementioned integrations levels were possible, because Colab works as a
53	-reverse proxy, therefore all external requests pass through it.
54	-
55	-Single Sign-On (SSO) is used to login users throughout all integrated
56	-applications. REMOTE\_USER HTTP header is sent to the applications and we
57	-expect that they know how to handle it, since this is a common authentication
58	-mechanism. The integrated applications should be on a local network to avoid
59	-security issues. With this the user will be able to navigate through the
60	-platform applications and will not be asked for authentication credentials.
61	-
62	-As Colab\footnote{\url{https://github.com/colab}} is a reverse proxy, it makes
63	-some HTML transformations in the HTTP response of integrated applications to
64	-provide a unified interface. Allows one to define some default templates to be
65	-used by all applications and overwrite when needed in its plugin. This approach
66	-allowed us to reuse some HTML pages which facilitates maintenance.
67	-
68	-A publish-subscribe implementation was used to handle events in the platform.
69	-Thus, if some application want to trigger some action in case that other
70	-application do something is possible. A registration of the desired events and
71	-implementation of the handlers must be done in the plugin of each application.
72	-This bring us many options to innovate in these integrations.
73	-
74	-A integrated search engine is provided by Colab. Each plugin specify which data
75	-will be indexed and will became available for the users, just need an simple
76	-implementation of how should perform the collection of data.	57	+Colab implements this integration by working as a reverse proxy for the
		58	+integration applications, that is, all external requests pass through
		59	+Colab before reaching them.
		60	+
		61	+Centralized authentication is handled by letting users authenticate to
		62	+Colab, which then sends a REMOTE\_USER HTTP header to applications,
		63	+which are expected to be pre-configured to accept that as an indication
		64	+of the current user (REMOTE\_USER is a standard authentication mechanism
		65	+for web applications). This allows users to be automatically identified
		66	+to each of the applications without having to enter credentials for each
		67	+of them.
		68	+%
		69	+Of course, this requires that the integrated applications are not
		70	+accessible on the public internet, otherwise malicious users could send
		71	+their own crafted REMOTE\_USER headers and impersonate any user.
		72	+
		73	+For the visual consistency, Colab is able to make transformations to the
		74	+HTML produced by the integrated applications, ir order to provide a
		75	+unified interface. It allows one to define default templates to be used
		76	+by all applications, as well as providing extra ones in a plugin. This
		77	+approach allowed us to reuse common HTML pages, what facilitates
		78	+maintenance.
		79	+
		80	+Colab also provides a publish-subscribe event system. This allows one of
		81	+the applications, or Colab itself, to trigger an action in another
		82	+application. For example, when a user registers with Colab, all
		83	+applications need to be notified in order to create their own internal
		84	+representation of that user account.
		85	+
		86	+Colab also provides an integrated search engine, which can be configured
		87	+to specify exactly what data needs to be indexed for each of the
		88	+applications, and how to direct the user to that piece of information on
		89	+the specific application.
77		90
78	Initially, Colab had support for a small set of applications (Trac, GNU	91	Initially, Colab had support for a small set of applications (Trac, GNU
79	-Mailman, and Apache Lucene) and all of them was hard-coded. Our team evolved
80	-Colab so that it can now receive plugins to add support for new applications
81	-with minimal changes to its existing core. We developed plugins to be used in
82	-the SPB platform: Noosfero, GitLab, and Mezuro.	92	+Mailman, and Apache Lucene) and support for them was hard-coded. Our
		93	+team evolved Colab so that it can now receive plugins that add support
		94	+new applications without the need of changes to the Colab core. We also
		95	+developed plugins to be used in the SPB platform: Noosfero, GitLab, and
		96	+Mezuro.
83		97
84	\subsection{Noosfero}	98	\subsection{Noosfero}
85		99
	@@ -92,38 +106,34 @@ home pages and documentation, and contact forms.		@@ -92,38 +106,34 @@ home pages and documentation, and contact forms.
92		106
93	\subsection{Gitlab}	107	\subsection{Gitlab}
94		108
95	-GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository manager
96	-with wiki pages and issue tracking features. On the one hand, Github is the
97	-most famous and used Git repository manager. On the other hand, it is not a
98	-FOSS. Gitlab is a FOSS platform and focuses on delivering a holistic solution
99	-that will see developers from idea to production seamlessly and on a single
100	-platform. Thus, we have worked on it to integrated to Colab.
101	-
102	-Moreover, Gitlab has several features working properly than Github that is
103	-interesting in SPB context such as free for private projects, built-in
104	-continuous integration and continuous deployment with best practices, more
105	-control during downtime and upgrade, flexible permissions, Work-in-Progress
106	-protection, move issues between projects, Group-level milestones, Create new
107	-branches from issues, Issue board, Time tracking, as well new features and
108	-improvements every month on the 22nd.	109	+GitLab\footnote{\url{http://gitlab.com}} is a web-based Git repository
		110	+manager with wiki pages and issue tracking features. Gitlab is a FOSS
		111	+platform and focuses on delivering a holistic solution that will see
		112	+developers from idea to production seamlessly and on a single platform.
		113	+
		114	+Gitlab has several unique features, such as built-in continuous
		115	+integration and continuous deployment, flexible permissions, tracking of
		116	+Work-in-Progress work, moving issues between projects, group-level
		117	+milestones, creating new branches from issues, issues board, and time
		118	+tracking.
109		119
110	\subsection{Mezuro}	120	\subsection{Mezuro}
111		121
112	Mezuro\footnote{\url{http://mezuro.org/}} is a platform to collect source code	122	Mezuro\footnote{\url{http://mezuro.org/}} is a platform to collect source code
113	-metric to monitor the internal quality of softwares written in C, C++,
114	-Java, Python, Ruby, and PHP. GNU Mailman is used for mailing lists.
115	-
116	-In general, source code metric tools also do not present a friendly way to
117	-interpret its results and, even more, do not follow a standardization between
118	-them. Mezuro brings and presents these results to
119	-the end user, specially, by analyzing source code metric history during its
120	-life cycle.
121	-
122	-The Mezuro platform provides a single interface
123	-grouping available tools, allows selection and composition of metrics in a
124	-flexible manner, stores the metrics evolution history, presents
125	-results in a friendly way, as well as, allows users to customize the given
126	-interpretation accordingly to their own context.	123	+metrics to monitor the internal quality of software written in C, C++,
		124	+Java, Python, Ruby, and PHP.
		125	+
		126	+In general, source code metrics tools also do not present a friendly way
		127	+to interpret its results and, even more, do not follow a standardization
		128	+between them. Mezuro collects and presents these results to the end
		129	+user, specially, by analyzing source code metric history during its life
		130	+cycle.
		131	+
		132	+The Mezuro platform provides a single interface grouping available
		133	+tools, allows selection and composition of metrics in a flexible manner,
		134	+stores the metrics evolution history, presents results in a friendly
		135	+way, as well as, allows users to customize the given interpretation
		136	+accordingly to their own context.
127		137
128	\subsection{System unification}	138	\subsection{System unification}
129		139