Download as
Browse Code »

Commit 3671984b24209b7a6c2516fcaf61d19bbd4c8b71

Authored by Sam Gleske
1 parent b75777fd
Exists in spb-stable and in 3 other branches 6-8-stable, 7-0-stable, 7-0-stable-spb

An architectural overview of GitLab 

This is to help new admins of GitLab understand what components
are involved in making GitLab work.
Showing 3 changed files with 128 additions and 17 deletions   Show diff stats
doc/development/architecture.md
  Diff comments   View file @3671984
1   -# GitLab project architecture
  1 +# GitLab Architecture Overview
  2 +---
2 3  
3   -GitLab project consists of two parts: GitLab and GitLab shell.
  4 +# Software delivery
4 5  
5   -## GitLab
  6 +There's two editions of GitLab: [Enterprise Edition](https://www.gitlab.com/features/) (EE) and [Community Edition](http://gitlab.org/gitlab-ce/) (CE). GitLab CE is delivered via git from the [gitlabhq repository](https://github.com/gitlabhq/gitlabhq/). New versions of GitLab are released in stable branches and the master branch is for bleeding edge development. EE releases are available not long after CE releases. To obtain the GitLab EE there is a [repository at gitlab.com](https://gitlab.com/subscribers/gitlab-ee). A new version of CE is delivered every month on the 22nd of the month. For this reason it is recommended to follow a monthly upgrade schedule because usually one can't skip versions when upgrading but must upgrade incrementally. Security updates come out on an informal basis.
6 7  
7   -Web application with background jobs workers.
8   -Provides you with UI and most of functionality.
9   -For some operations like repo creation - uses GitLab shell.
  8 +Both EE and CE require an add-on component called gitlab-shell. It is obtained from the [gitlab-shell repository](https://github.com/gitlabhq/gitlab-shell). New versions are usually tags but staying on the master branch will give you the latest stable version. New releases are generally around the same time as GitLab CE releases with exception for informal security updates deemed critical.
10 9  
11   -Uses:
12   - * Ruby as main language for application code and most libraries.
13   - * [Rails](http://rubyonrails.org/) web framework as main framework for application.
14   - * Mysql or postgres as main databases. Used for persistent data storage(users, project, issues etc).
15   - * Redis database. Used for cache and exchange data between some components.
16   - * Python2 because of [pygments](http://pygments.org/) as code syntax highlighter.
  10 +# System Layout
17 11  
18   -## GitLab shell
  12 +When referring to `~git` it means the home directory of the `git` user which is typically `/home/git`.
19 13  
20   -Command line ruby application. Used by GitLab through shell commands.
21   -It provides interface to all kind of manipulations with repositories and ssh keys.
22   -Full list of commands you can find in README of GitLab shell repo.
23   -Works on pure ruby and do not require any additional software.
  14 +GitLab is primarily installed within the `/home/git` user home directory as `git` user. Within the home directory is where the gitlabhq server software resides as well as the repositories (though repository location is configurable). The bare repositories are located in `~git/repositories`. GitLab is a ruby on rails application so the particulars of the inner workings can be learned by studying how a ruby on rails application works. To serve repositories over SSH there's an add-on application called gitlab-shell which is installed in `/home/git/gitlab-shell`.
  15 +
  16 +## Components
  17 +
  18 +![GitLab Diagram Overview](resources/gitlab_diagram_overview.png "GitLab Diagram Overview")
  19 +
  20 +A typical install of GitLab will be on RHEL or Ubuntu Linux. It uses Apache or nginx as a web front end to proxypass the Unicorn web server. Communication between Unicorn and the front end is usually HTTP but access via socket is also supported. The web front end accesses `~git/gitlab/public` bypassing the Unicorn server to serve static pages, attachments, and other resources the GitLab core creates (such as uploaded avatars or archives). GitLab serves web pages and a [GitLab API](https://github.com/gitlabhq/gitlabhq/tree/master/doc/api) using the Unicorn web server. It uses Sidekiq as a job queue which, in turn, uses redis as a non-persistent database backend for job information, meta data, and incomming jobs. The GitLab web app uses MySQL or PostgreSQL for persistent database information (e.g. users, permissions, issues, other meta data). GitLab stores the bare git repositories it serves in `~git/repositories` by default. It also keeps default branch and hook information with the bare repository. GitLab maintains a checked out version of each repository in `~git/gitlab-satellites`. The satellite repository is used by the web interface for editing repositories and the wiki which is also a git repository. When serving repositories over HTTP/HTTPS GitLab utilizes the GitLab API to resolve authorization and access as well as serving git objects.
  21 +
  22 +The add-on component gitlab-shell serves repositories over SSH. It manages the SSH keys within `~git/.ssh/authorized_keys` which should not be manually edited. gitlab-shell accesses the bare repositories directly to serve git objects and communicates with redis to submit jobs to Sidekiq for GitLab to process. gitlab-shell queries the GitLab API to determine authorization and access.
  23 +
  24 +## Processes
  25 +
  26 + ps aux | grep '^git'
  27 +
  28 +GitLab has several components to operate. As a system user (i.e. any user that is not the `git` user) it requires a persistent database (MySQL/PostreSQL) and redis database. It also uses Apache httpd or nginx to proxypass Unicorn. As the `git` user it starts Sidekiq and Unicorn (a simple ruby HTTP server running on port `8080` by default). Under the gitlab user there are normally 6 processes: `unicorn_rails master` (1 process), `unicorn_rails worker` (2 processes), `python pygments` (2 processes), `sidekiq` (1 process). Pygments is used by GitLab for syntax highlighting in the web interface.
  29 +
  30 +## Repository access
  31 +
  32 +Repositories get accessed via HTTP or SSH. HTTP cloning/push/pull utilizes the GitLab API and SSH cloning is handled by gitlab-shell (previously explained).
  33 +
  34 +# Troubleshooting
  35 +
  36 +See also the [IRC F.A.Q.](https://github.com/gitlabhq/gitlab-public-wiki/wiki/IRC-channel-Guidelines-and-F.A.Q.) and [Troubleshooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki).
  37 +
  38 +## Services
  39 +
  40 +GitLab (includes Unicorn and Sidekiq), redis (non-persistent DB), SSH (all of the following)
  41 +
  42 +```
  43 +/etc/init.d/gitlab
  44 +Usage: service gitlab {start|stop|restart|reload|status}
  45 +
  46 +/etc/init.d/redis
  47 +Usage: /etc/init.d/redis {start|stop|status|restart|condrestart|try-restart}
  48 +
  49 +/etc/init.d/sshd
  50 +Usage: /etc/init.d/sshd {start|stop|restart|reload|force-reload|condrestart|try-restart|status}
  51 +```
  52 +
  53 +Web front end (one of the following)
  54 +
  55 +```
  56 +/etc/init.d/httpd
  57 +Usage: httpd {start|stop|restart|condrestart|try-restart|force-reload|reload|status|fullstatus|graceful|help|configtest}
  58 +
  59 +$ /etc/init.d/nginx
  60 +Usage: nginx {start|stop|restart|reload|force-reload|status|configtest}
  61 +```
  62 +
  63 +Persistent database (one of the following)
  64 +
  65 +```
  66 +/etc/init.d/mysqld
  67 +Usage: /etc/init.d/mysqld {start|stop|status|restart|condrestart|try-restart|reload|force-reload}
  68 +
  69 +$ /etc/init.d/postgresql
  70 +Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [version ..]
  71 +```
  72 +
  73 +## Log locations
  74 +
  75 +Note: `~git/` is shorthand for `/home/git`.
  76 +
  77 +gitlabhq (includes Unicorn and Sidekiq logs)
  78 +
  79 +* `~git/gitlab/log/` contains `application.log`, `production.log`, `sidekiq.log`, `unicorn.stdout.log`, `githost.log`, `satellites.log`, and `unicorn.stderr.log` normally.
  80 +
  81 +gitlab-shell
  82 +
  83 +* `~git/gitlab-shell/gitlab-shell.log`
  84 +
  85 +ssh
  86 +
  87 +* `/var/log/auth.log` auth log (on Ubuntu).
  88 +* `/var/log/secure` auth log (on RHEL).
  89 +
  90 +nginx
  91 +
  92 +* `/var/log/nginx/` contains error and access logs.
  93 +
  94 +Apache httpd
  95 +
  96 +* [Explanation of apache logs](http://httpd.apache.org/docs/2.2/logs.html).
  97 +* `/var/log/apache2/` contains error and output logs (on Ubuntu).
  98 +* `/var/log/httpd/` contains error and output logs (on RHEL).
  99 +
  100 +redis
  101 +
  102 +* `/var/log/redis/redis.log` there are also logrotated logs there.
  103 +
  104 +PostgreSQL
  105 +
  106 +* `/var/log/postgresql/*`
  107 +
  108 +MySQL
  109 +
  110 +* `/var/log/mysql/*`
  111 +* `/var/log/mysql.*`
  112 +
  113 +## GitLab specific config files
  114 +
  115 +GitLab has configuration files located in `~git/gitlab/config/*`. Commonly referenced config files include:
  116 +
  117 +* `gitlab.yml` - GitLab configuration.
  118 +* `unicorn.rb` - Unicorn web server settings.
  119 +* `database.yml` - Database connection settings.
  120 +
  121 +gitlab-shell has a configuration file at `~git/gitlab-shell/config.yml`.
  122 +
  123 +## Maintenance Tasks
  124 +
  125 +[gitlabhq](https://github.com/gitlabhq/gitlabhq) provides rake tasks with which you see version information and run a quick check on your configuration to ensure it is configured properly within the application. See [maintenance rake tasks](https://github.com/gitlabhq/gitlabhq/blob/master/doc/raketasks/maintenance.md). In a nutshell, do the following:
  126 +
  127 +```
  128 +sudo -i -u git
  129 +cd gitlab
  130 +bundle exec rake gitlab:env:info RAILS_ENV=production
  131 +bundle exec rake gitlab:check RAILS_ENV=production
  132 +```
  133 +
  134 +Note: It is recommended to log into the `git` user using `sudo -i -u git` or `sudo su - git`. While the sudo commands provided by gitlabhq work in Ubuntu they do not always work in RHEL.
... ...
doc/development/resources/gitlab_diagram_overview.odg 0 → 100644
View file @3671984
No preview for this file type
doc/development/resources/gitlab_diagram_overview.png 0 → 100644
View file @3671984

127 KB