Commit eae41ad1df37cec184179df02fce7faa9434cb2a

Authored by Sytse Sijbrandij
1 parent b2b88b2f

Non-interactive AWS install by running a single script.

Non-interactive AWS install by running a single script.

Complete non-interactive AWS install with one line.

Squash of 28 commits:

Initial commit of complete install script.

Must be sudo.

Non interactive prostgres.

Install mysql with unique password.

Service file.

Add init file.

Nginx config.

Script contains aws specific settings.

Update references after rename.

Prevent localhost ssh prompt.

Postfix with policyd.

Typo in localhost. Postfix package has other name. sudo -l session needs oneliners.

More sudo commands to prevent permission problems.

Change in existing file.

Change in existing file must be referenced.

Switch order.

Postfix comment.

Old way to generate password, less sudo.

todo login

Must use private ip and login instructions.

Starting up.

Fixed and checked various todos.

Use service. Set host name. Go to default dir.

Only works on next login.

Only replace first host.

Cleaned up the comments.

Link to nginx config.

Move to aws dir.

Update links to aws for interactive install.

Existing script stays at the same location.

Change directory. Use the new init scripts for gitlab service and nginx. Use the branch for file downloads. Update with newest changes to installation.md.

Correct reference.

Use master branch since we need the nginx condfig and use absolute paths.

Need to copy with sudo.
doc/debian_ubuntu.sh
... ... @@ -3,7 +3,8 @@
3 3 sudo apt-get update
4 4 sudo apt-get upgrade
5 5  
6   -sudo apt-get install -y git git-core wget curl gcc checkinstall libxml2-dev libxslt-dev sqlite3 libsqlite3-dev libcurl4-openssl-dev libreadline-gplv2-dev libc6-dev libssl-dev libmysql++-dev make build-essential zlib1g-dev libicu-dev redis-server openssh-server python-dev python-pip libyaml-dev postfix
  6 +sudo DEBIAN_FRONTEND='noninteractive' apt-get install -y postfix-policyd-spf-python # Install postfix without prompting.
  7 +sudo apt-get install -y git git-core wget curl gcc checkinstall libxml2-dev libxslt-dev sqlite3 libsqlite3-dev libcurl4-openssl-dev libreadline-gplv2-dev libc6-dev libssl-dev libmysql++-dev make build-essential zlib1g-dev libicu-dev redis-server openssh-server python-dev python-pip libyaml-dev
7 8  
8 9 wget http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p194.tar.gz
9 10 tar xfvz ruby-1.9.3-p194.tar.gz
... ...
doc/installation.md
1   -## Platform requirements:
  1 +## Platform requirements:
2 2  
3 3 **The project is designed for the Linux operating system.**
4 4  
... ... @@ -22,7 +22,7 @@ You might have some luck using these, but no guarantees:
22 22  
23 23 Gitlab does **not** run on Windows and we have no plans of making Gitlab compatible.
24 24  
25   -## This installation guide created for Debian/Ubuntu and properly tested.
  25 +## This installation guide created for Debian/Ubuntu and properly tested.
26 26  
27 27 The installation consists of 6 steps:
28 28  
... ... @@ -43,13 +43,13 @@ Also read the [Read this before you submit an issue](https://github.com/gitlabhq
43 43  
44 44 > - - -
45 45 > First 3 steps can be easily skipped with simply install script:
46   ->
47   -> # Install curl and sudo
  46 +>
  47 +> # Install curl and sudo
48 48 > apt-get install curl sudo
49   ->
  49 +>
50 50 > # 3 steps in 1 command :)
51 51 > curl https://raw.github.com/gitlabhq/gitlabhq/master/doc/debian_ubuntu.sh | sh
52   ->
  52 +>
53 53 > Now you can go to step 4"
54 54 > - - -
55 55  
... ... @@ -61,7 +61,7 @@ Also read the [Read this before you submit an issue](https://github.com/gitlabhq
61 61 sudo apt-get upgrade
62 62  
63 63 sudo apt-get install -y wget curl gcc checkinstall libxml2-dev libxslt-dev sqlite3 libsqlite3-dev libcurl4-openssl-dev libreadline6-dev libc6-dev libssl-dev libmysql++-dev make build-essential zlib1g-dev libicu-dev redis-server openssh-server git-core python-dev python-pip libyaml-dev postfix
64   -
  64 +
65 65 # If you want to use MySQL:
66 66 sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev
67 67  
... ... @@ -77,7 +77,7 @@ Also read the [Read this before you submit an issue](https://github.com/gitlabhq
77 77 # 3. Install gitolite
78 78  
79 79 Create user for git:
80   -
  80 +
81 81 sudo adduser \
82 82 --system \
83 83 --shell /bin/sh \
... ... @@ -90,7 +90,7 @@ Create user for git:
90 90 Create user for gitlab:
91 91  
92 92 # ubuntu/debian
93   - sudo adduser --disabled-login --gecos 'gitlab system' gitlab
  93 + sudo adduser --disabled-login --gecos 'gitlab system' gitlab
94 94  
95 95 Add your user to git group:
96 96  
... ... @@ -103,7 +103,7 @@ Generate key:
103 103 Get gitolite source code:
104 104  
105 105 cd /home/git
106   - sudo -H -u git git clone git://github.com/gitlabhq/gitolite /home/git/gitolite
  106 + sudo -H -u git git clone git://github.com/gitlabhq/gitolite /home/git/gitolite
107 107  
108 108 Setup:
109 109  
... ... @@ -114,21 +114,21 @@ Setup:
114 114  
115 115 sudo -u git -H sed -i 's/0077/0007/g' /home/git/share/gitolite/conf/example.gitolite.rc
116 116 sudo -u git -H sh -c "PATH=/home/git/bin:$PATH; gl-setup -q /home/git/gitlab.pub"
117   -
  117 +
118 118 Permissions:
119 119  
120 120 sudo chmod -R g+rwX /home/git/repositories/
121 121 sudo chown -R git:git /home/git/repositories/
122   - sudo chown gitlab:gitlab /home/git/repositories/**/hooks/post-receive
  122 + sudo chown gitlab:gitlab /home/git/repositories/**/hooks/post-receive
123 123  
124 124 #### CHECK: Logout & login again to apply git group to your user
125   -
  125 +
126 126 # clone admin repo to add localhost to known_hosts
127 127 # & be sure your user has access to gitolite
128   - sudo -u gitlab -H git clone git@localhost:gitolite-admin.git /tmp/gitolite-admin
  128 + sudo -u gitlab -H git clone git@localhost:gitolite-admin.git /tmp/gitolite-admin
129 129  
130 130 # if succeed you can remove it
131   - sudo rm -rf /tmp/gitolite-admin
  131 + sudo rm -rf /tmp/gitolite-admin
132 132  
133 133 **IMPORTANT! If you cant clone `gitolite-admin` repository - DONT PROCEED INSTALLATION**
134 134  
... ... @@ -140,7 +140,7 @@ Permissions:
140 140 cd /home/gitlab
141 141 sudo -H -u gitlab git clone -b stable git://github.com/gitlabhq/gitlabhq.git gitlab
142 142 cd gitlab
143   -
  143 +
144 144 sudo -u gitlab mkdir tmp
145 145  
146 146 # Rename config files
... ... @@ -151,22 +151,22 @@ Permissions:
151 151 # SQLite
152 152 sudo -u gitlab cp config/database.yml.sqlite config/database.yml
153 153  
154   - # Or
  154 + # Or
155 155 # Mysql
156 156 # Install MySQL as directed in Step #1
157   -
  157 +
158 158 # Login to MySQL
159   - $ mysql -u root -p
160   -
  159 + $ mysql -u root -p
  160 +
161 161 # Create the gitlabhq production database
162 162 mysql> CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;
163   -
  163 +
164 164 # Create the MySQL User change $password to a real password
165   - mysql> CREATE USER 'gitlab'@'localhost' IDENTIFIED BY '$password';
166   -
  165 + mysql> CREATE USER 'gitlab'@'localhost' IDENTIFIED BY '$password';
  166 +
167 167 # Grant proper permissions to the MySQL User
168 168 mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON `gitlabhq_production`.* TO 'gitlab'@'localhost';
169   -
  169 +
170 170 # Exit MySQL Server and copy the example config, make sure to update username/password in config/database.yml
171 171 sudo -u gitlab cp config/database.yml.example config/database.yml
172 172  
... ... @@ -177,7 +177,7 @@ Permissions:
177 177 #### Setup DB
178 178  
179 179 sudo -u gitlab bundle exec rake gitlab:app:setup RAILS_ENV=production
180   -
  180 +
181 181 Checking status:
182 182  
183 183 sudo -u gitlab bundle exec rake gitlab:app:status RAILS_ENV=production
... ... @@ -197,13 +197,13 @@ Checking status:
197 197 Can clone gitolite-admin?............YES
198 198 UMASK for .gitolite.rc is 0007? ............YES
199 199  
200   -If you got all YES - congrats! You can go to next step.
  200 +If you got all YES - congrats! You can go to next step.
201 201  
202 202 # 5. Server up
203 203  
204 204 Application can be started with next command:
205 205  
206   - # For test purposes
  206 + # For test purposes
207 207 sudo -u gitlab bundle exec rails s -e production
208 208  
209 209 # As daemon
... ... @@ -239,7 +239,7 @@ You can login via web using admin generated with setup:
239 239 sudo -u gitlab cp config/unicorn.rb.orig config/unicorn.rb
240 240 sudo -u gitlab bundle exec unicorn_rails -c config/unicorn.rb -E production -D
241 241  
242   -Edit /etc/nginx/nginx.conf. In the *http* section add:
  242 +Edit /etc/nginx/nginx.conf. In the *http* section add the following section of code or replace it completely with https://raw.github.com/dosire/gitlabhq/master/aws/nginx.conf
243 243  
244 244 upstream gitlab {
245 245 server unix:/home/gitlab/gitlab/tmp/sockets/gitlab.socket;
... ... @@ -249,27 +249,27 @@ Edit /etc/nginx/nginx.conf. In the *http* section add:
249 249 listen YOUR_SERVER_IP:80; # e.g., listen 192.168.1.1:80;
250 250 server_name YOUR_SERVER_FQDN; # e.g., server_name source.example.com;
251 251 root /home/gitlab/gitlab/public;
252   -
  252 +
253 253 # individual nginx logs for this gitlab vhost
254 254 access_log /var/log/nginx/gitlab_access.log;
255 255 error_log /var/log/nginx/gitlab_error.log;
256   -
  256 +
257 257 location / {
258 258 # serve static files from defined root folder;.
259 259 # @gitlab is a named location for the upstream fallback, see below
260 260 try_files $uri $uri/index.html $uri.html @gitlab;
261 261 }
262   -
263   - # if a file, which is not found in the root folder is requested,
  262 +
  263 + # if a file, which is not found in the root folder is requested,
264 264 # then the proxy pass the request to the upsteam (gitlab unicorn)
265 265 location @gitlab {
266 266 proxy_redirect off;
267   -
  267 +
268 268 # you need to change this to "https", if you set "ssl" directive to "on"
269 269 proxy_set_header X-FORWARDED_PROTO http;
270 270 proxy_set_header Host $http_host;
271 271 proxy_set_header X-Real-IP $remote_addr;
272   -
  272 +
273 273 proxy_pass http://gitlab;
274 274 }
275 275 }
... ... @@ -292,7 +292,7 @@ Create init script in /etc/init.d/gitlab:
292 292 # Short-Description: GitLab git repository management
293 293 # Description: GitLab git repository management
294 294 ### END INIT INFO
295   -
  295 +
296 296 DAEMON_OPTS="-c /home/gitlab/gitlab/config/unicorn.rb -E production -D"
297 297 NAME=unicorn
298 298 DESC="Gitlab service"
... ...
lib/support/aws/debian_ubuntu_aws.sh 0 → 100644
... ... @@ -0,0 +1,125 @@
  1 +#!/bin/sh
  2 +
  3 +# ABOUT
  4 +# This script performs a complete installation of Gitlab (master branch).
  5 +# Is can be run with one command without needing _any_ user input after that.
  6 +# This script only works on Amazon Web Services (AWS).
  7 +# The operating system used is Ubuntu 12.04 64bit.
  8 +
  9 +# TODO
  10 +# @dosire will send a pull request after this is merged in to change dosire/gitlabhq/non-interactive-aws-install links to gitlabhq/gitlabhq/master and reference this script from installation.md
  11 +
  12 +# HOWTO
  13 +# Signup for AWS, free tier are available at http://aws.amazon.com/free/
  14 +# Go to EC2 tab in the AWS console EC2 https://console.aws.amazon.com/ec2/home
  15 +# Click the 'Launch Instance' button
  16 +# Select: 'Quick launch wizard' and continue
  17 +# Choose a key pair => Create New => Name it => Download it
  18 +# Choose a Launch Configuration => Select 'More Amazon Marketplace Images'
  19 +# Press 'Continue'
  20 +# Enter 'ubuntu/images/ubuntu-precise-12.04-amd64-server-20120424' and press 'Search'
  21 +# Select the only result (ami-3c994355) and press 'Continue'
  22 +# Press 'Edit details' if you want to modify something, for example make the type 'c1.medium' to make the install faster.
  23 +# Press the 'Launch' button
  24 +# Press 'Close'
  25 +# Click 'Security Groups' under the left hand menu 'NETWORK & SECURITY'
  26 +# Select the newly create seciruty group, probably named 'quicklaunch-1'
  27 +# Click on the Inbound tab
  28 +# In the 'Create a new rule' dropdown select 'HTTP'
  29 +# Press 'Add Rule'
  30 +# In the 'Create a new rule' dropdown select 'HTTPS'
  31 +# Press 'Add Rule'
  32 +# Press 'Apply Rule Changes'
  33 +# Give the following command in your local terminal while suptituting the UPPERCASE items
  34 +# 'ssh -i LOCATION_OF_AWS_KEY_PAIR_PRIVATE_KEY PUBLIC_DNS_OF_THE_NEW_SERVER'
  35 +# Execute the curl command below and when its ready follow the printed 'Log in instuctions'
  36 +# curl https://raw.github.com/dosire/gitlabhq/non-interactive-aws-install/lib/support/aws/debian_ubuntu_aws.sh | sh
  37 +
  38 +# Prevent fingerprint prompt for localhost in step 1 to 3.
  39 +echo "Host localhost
  40 + StrictHostKeyChecking no
  41 + UserKnownHostsFile=/dev/null" | sudo tee -a /etc/ssh/ssh_config
  42 +
  43 +# Existing script for Step 1 to 3
  44 +curl https://raw.github.com/dosire/gitlabhq/non-interactive-aws-install/doc/debian_ubuntu.sh | sh
  45 +
  46 +# Install MySQL
  47 +sudo apt-get install -y makepasswd # Needed to create a unique password non-interactively.
  48 +userPassword=$(makepasswd --char=10) # Generate a random MySQL password
  49 +# Note that the lines below creates a cleartext copy of the random password in /var/cache/debconf/passwords.dat
  50 +# This file is normally only readable by root and the password will be deleted by the package management system after install.
  51 +echo mysql-server mysql-server/root_password password $userPassword | sudo debconf-set-selections
  52 +echo mysql-server mysql-server/root_password_again password $userPassword | sudo debconf-set-selections
  53 +sudo apt-get install -y mysql-server
  54 +
  55 +# Gitlab install
  56 +sudo gem install charlock_holmes --version '0.6.8'
  57 +sudo pip install pygments
  58 +sudo gem install bundler
  59 +sudo su -l gitlab -c "git clone git://github.com/gitlabhq/gitlabhq.git gitlab" # Using master everywhere.
  60 +sudo su -l gitlab -c "cd gitlab && mkdir tmp"
  61 +sudo su -l gitlab -c "cd gitlab/config && cp gitlab.yml.example gitlab.yml"
  62 +sudo su -l gitlab -c "cd gitlab/config && cp database.yml.example database.yml"
  63 +sudo sed -i 's/"secure password"/"'$userPassword'"/' /home/gitlab/gitlab/config/database.yml # Insert the mysql root password.
  64 +sudo su -l gitlab -c "cd gitlab && bundle install --without development test --deployment"
  65 +sudo su -l gitlab -c "cd gitlab && bundle exec rake gitlab:app:setup RAILS_ENV=production"
  66 +
  67 +# Setup gitlab hooks
  68 +sudo cp /home/gitlab/gitlab/lib/hooks/post-receive /home/git/share/gitolite/hooks/common/post-receive
  69 +sudo chown git:git /home/git/share/gitolite/hooks/common/post-receive
  70 +
  71 +# Set the first occurrence of host in the Gitlab config to the publicly available domain name
  72 +sudo sed -i '0,/host/s/localhost/'`wget -qO- http://instance-data/latest/meta-data/public-hostname`'/' /home/gitlab/gitlab/config/gitlab.yml
  73 +
  74 +# Gitlab installation test (optional)
  75 +# sudo -u gitlab bundle exec rake gitlab:app:status RAILS_ENV=production
  76 +# sudo -u gitlab bundle exec rails s -e production
  77 +# sudo -u gitlab bundle exec rake environment resque:work QUEUE=* RAILS_ENV=production BACKGROUND=no
  78 +
  79 +# Install and configure Nginx
  80 +sudo apt-get install -y nginx
  81 +sudo cp /home/gitlab/gitlab/lib/support/nginx-gitlab /etc/nginx/sites-available/gitlab
  82 +sudo ln -s /etc/nginx/sites-available/gitlab /etc/nginx/sites-enabled/gitlab
  83 +sudo sed -i 's/YOUR_SERVER_IP/'`wget -qO- http://instance-data/latest/meta-data/local-ipv4`'/' /etc/nginx/sites-available/gitlab # Set private ip address (public won't work).
  84 +sudo sed -i 's/YOUR_SERVER_FQDN/'`wget -qO- http://instance-data/latest/meta-data/public-hostname`'/' /etc/nginx/sites-available/gitlab # Set public dns domain name.
  85 +
  86 +# Configure Unicorn
  87 +sudo -u gitlab cp /home/gitlab/gitlab/config/unicorn.rb.orig /home/gitlab/gitlab/config/unicorn.rb
  88 +
  89 +# Create a Gitlab service
  90 +sudo cp /home/gitlab/gitlab/lib/support/init-gitlab /etc/init.d/gitlab
  91 +sudo chmod +x /etc/init.d/gitlab && sudo update-rc.d gitlab defaults
  92 +
  93 +## Gitlab service commands (unicorn and resque)
  94 +## restart doesn't restart resque, only start/stop effect it.
  95 +sudo -u gitlab service gitlab start
  96 +# sudo -u gitlab service gitlab restart
  97 +# sudo -u gitlab service gitlab stop
  98 +
  99 +# nginx Service commands
  100 +# sudo service nginx start
  101 +sudo service nginx restart
  102 +# sudo service nginx stop
  103 +
  104 +# Manual startup commands for troubleshooting when the service commands do not work
  105 +# sudo -u gitlab bundle exec unicorn_rails -c config/unicorn.rb -E production -D
  106 +# sudo su -l gitlab -c "cd gitlab && ./resque.sh"
  107 +
  108 +# Monitoring commands
  109 +# sudo tail -f /var/log/nginx/access.log;
  110 +# sudo tail -f /var/log/nginx/error.log;
  111 +
  112 +# Go to gitlab directory by default on next login.
  113 +echo 'cd /home/gitlab/gitlab' >> /home/ubuntu/.bashrc
  114 +
  115 +echo ''
  116 +echo '###########################################'
  117 +echo '# Log in instuctions #'
  118 +echo '###########################################'
  119 +echo ''
  120 +echo "Surf to this Gitlab installation in your browser:"
  121 +echo "http://`wget -qO- http://instance-data/latest/meta-data/public-hostname`/"
  122 +echo ''
  123 +echo 'and login with the following Email and Password:'
  124 +echo 'admin@local.host'
  125 +echo '5iveL!fe'
0 126 \ No newline at end of file
... ...