Download as
Browse Code »

Commit 2838abd63378b3b546ec63248505a6c81db88332

Authored by Sytse Sijbrandij
2 parents 81d759e9 c7ce512b
Exists in master and in 9 other branches 7-0-stable, 7-0-stable-ee, 7-0-stable-spb, 7-1-stable, 7-1-stable-ee, 7-2-stable, 7-2-stable-ee, branch_test, nginx_ssl_tweaks

Merge branch 'runit_log_settings' into 'master' 

Make Runit log rotation configurable
Showing 15 changed files with 110 additions and 11 deletions   Show diff stats
@@ -17,6 +17,8 @@ omnibus-gitlab repository. @@ -17,6 +17,8 @@ omnibus-gitlab repository.
17 - Include Python and docutils for reStructuredText support 17 - Include Python and docutils for reStructuredText support
18 - Update Ruby to version 2.1.1 18 - Update Ruby to version 2.1.1
19 - Update Git to version 2.0.0 19 - Update Git to version 2.0.0
  20 +- Make Runit log rotation configurable
  21 +- Change default Runit log rotation from 10x1MB to 30x24h
20 22
21 6.9.2 23 6.9.2
22 - Create the authorized-keys.lock file for gitlab-shell 1.9.4 24 - Create the authorized-keys.lock file for gitlab-shell 1.9.4
@@ -455,6 +455,45 @@ Omnibus-gitlab uses four different directories. @@ -455,6 +455,45 @@ Omnibus-gitlab uses four different directories.
455 - `/var/log/gitlab` contains all log data generated by components of 455 - `/var/log/gitlab` contains all log data generated by components of
456 omnibus-gitlab. 456 omnibus-gitlab.
457 457
  458 +## Logs
  459 +
  460 +### Tail logs in a console on the server
  461 +
  462 +If you want to 'tail', i.e. view live log updates of GitLab logs you can use
  463 +`gitlab-ctl tail`.
  464 +
  465 +```shell
  466 +# Tail all logs; press Ctrl-C to exit
  467 +sudo gitlab-ctl tail
  468 +
  469 +# Drill down to a sub-directory of /var/log/gitlab
  470 +sudo gitlab-ctl tail gitlab-rails
  471 +
  472 +# Drill down to an individual file
  473 +sudo gitlab-ctl tail nginx/gitlab_error.log
  474 +```
  475 +
  476 +### Runit logs
  477 +
  478 +The Runit-managed services in omnibus-gitlab generate log data using
  479 +[svlogd][svlogd]. See the [svlogd documentation][svlogd] for more information
  480 +about the files it generates.
  481 +
  482 +You can modify svlogd settings via `/etc/gitlab/gitlab.rb` with the following settings:
  483 +
  484 +```ruby
  485 +# Below are the default values
  486 +logging['svlogd_size'] = 200 * 1024 * 1024 # rotate after 200 MB of log data
  487 +logging['svlogd_num'] = 30 # keep 30 rotated log files
  488 +logging['svlogd_timeout'] = 24 * 60 * 60 # rotate after 24 hours
  489 +logging['svlogd_filter'] = "gzip" # compress logs with gzip
  490 +logging['svlogd_udp'] = nil # transmit log messages via UDP
  491 +logging['svlogd_prefix'] = nil # custom prefix for log messages
  492 +
  493 +# Optionally, you can override the prefix for e.g. Nginx
  494 +nginx['svlogd_prefix'] = "nginx"
  495 +```
  496 +
458 ## Starting a Rails console session 497 ## Starting a Rails console session
459 498
460 If you need access to a Rails production console for your GitLab installation 499 If you need access to a Rails production console for your GitLab installation
@@ -543,3 +582,4 @@ This omnibus installer project is based on the awesome work done by Chef in @@ -543,3 +582,4 @@ This omnibus installer project is based on the awesome work done by Chef in
543 [database.yml.postgresql]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/database.yml.postgresql 582 [database.yml.postgresql]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/database.yml.postgresql
544 [database.yml.mysql]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/database.yml.mysql 583 [database.yml.mysql]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/database.yml.mysql
545 [gitlab.yml.example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example 584 [gitlab.yml.example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example
  585 +[svlogd]: http://smarden.org/runit/svlogd.8.html
files/gitlab-cookbooks/gitlab/attributes/default.rb
  Diff comments   View file @2838abd
@@ -175,8 +175,6 @@ default['gitlab']['postgresql']['ha'] = false @@ -175,8 +175,6 @@ default['gitlab']['postgresql']['ha'] = false
175 default['gitlab']['postgresql']['dir'] = "/var/opt/gitlab/postgresql" 175 default['gitlab']['postgresql']['dir'] = "/var/opt/gitlab/postgresql"
176 default['gitlab']['postgresql']['data_dir'] = "/var/opt/gitlab/postgresql/data" 176 default['gitlab']['postgresql']['data_dir'] = "/var/opt/gitlab/postgresql/data"
177 default['gitlab']['postgresql']['log_directory'] = "/var/log/gitlab/postgresql" 177 default['gitlab']['postgresql']['log_directory'] = "/var/log/gitlab/postgresql"
178 -default['gitlab']['postgresql']['svlogd_size'] = 1000000  
179 -default['gitlab']['postgresql']['svlogd_num'] = 10  
180 default['gitlab']['postgresql']['username'] = "gitlab-psql" 178 default['gitlab']['postgresql']['username'] = "gitlab-psql"
181 default['gitlab']['postgresql']['uid'] = nil 179 default['gitlab']['postgresql']['uid'] = nil
182 default['gitlab']['postgresql']['gid'] = nil 180 default['gitlab']['postgresql']['gid'] = nil
@@ -217,8 +215,6 @@ default['gitlab']['redis']['enable'] = true @@ -217,8 +215,6 @@ default['gitlab']['redis']['enable'] = true
217 default['gitlab']['redis']['ha'] = false 215 default['gitlab']['redis']['ha'] = false
218 default['gitlab']['redis']['dir'] = "/var/opt/gitlab/redis" 216 default['gitlab']['redis']['dir'] = "/var/opt/gitlab/redis"
219 default['gitlab']['redis']['log_directory'] = "/var/log/gitlab/redis" 217 default['gitlab']['redis']['log_directory'] = "/var/log/gitlab/redis"
220 -default['gitlab']['redis']['svlogd_size'] = 1000000  
221 -default['gitlab']['redis']['svlogd_num'] = 10  
222 default['gitlab']['redis']['username'] = "gitlab-redis" 218 default['gitlab']['redis']['username'] = "gitlab-redis"
223 default['gitlab']['redis']['uid'] = nil 219 default['gitlab']['redis']['uid'] = nil
224 default['gitlab']['redis']['gid'] = nil 220 default['gitlab']['redis']['gid'] = nil
@@ -254,3 +250,13 @@ default['gitlab']['nginx']['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqd @@ -254,3 +250,13 @@ default['gitlab']['nginx']['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqd
254 default['gitlab']['nginx']['ssl_ciphers'] = "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4" 250 default['gitlab']['nginx']['ssl_ciphers'] = "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4"
255 default['gitlab']['nginx']['ssl_prefer_server_ciphers'] = "on" 251 default['gitlab']['nginx']['ssl_prefer_server_ciphers'] = "on"
256 default['gitlab']['nginx']['listen_address'] = '*' 252 default['gitlab']['nginx']['listen_address'] = '*'
  253 +
  254 +###
  255 +# Logging
  256 +###
  257 +default['gitlab']['logging']['svlogd_size'] = 200 * 1024 * 1024 # rotate after 200 MB of log data
  258 +default['gitlab']['logging']['svlogd_num'] = 30 # keep 30 rotated log files
  259 +default['gitlab']['logging']['svlogd_timeout'] = 24 * 60 * 60 # rotate after 24 hours
  260 +default['gitlab']['logging']['svlogd_filter'] = "gzip" # compress logs with gzip
  261 +default['gitlab']['logging']['svlogd_udp'] = nil # transmit log messages via UDP
  262 +default['gitlab']['logging']['svlogd_prefix'] = nil # custom prefix for log messages
files/gitlab-cookbooks/gitlab/libraries/gitlab.rb
  Diff comments   View file @2838abd
@@ -43,6 +43,7 @@ module Gitlab @@ -43,6 +43,7 @@ module Gitlab
43 unicorn Mash.new 43 unicorn Mash.new
44 sidekiq Mash.new 44 sidekiq Mash.new
45 nginx Mash.new 45 nginx Mash.new
  46 + logging Mash.new
46 node nil 47 node nil
47 external_url nil 48 external_url nil
48 git_data_dir nil 49 git_data_dir nil
@@ -134,6 +135,7 @@ module Gitlab @@ -134,6 +135,7 @@ module Gitlab
134 "unicorn", 135 "unicorn",
135 "sidekiq", 136 "sidekiq",
136 "nginx", 137 "nginx",
  138 + "logging",
137 "postgresql" 139 "postgresql"
138 ].each do |key| 140 ].each do |key|
139 rkey = key.gsub('_', '-') 141 rkey = key.gsub('_', '-')
files/gitlab-cookbooks/gitlab/recipes/nginx.rb
  Diff comments   View file @2838abd
@@ -71,6 +71,7 @@ runit_service "nginx" do @@ -71,6 +71,7 @@ runit_service "nginx" do
71 options({ 71 options({
72 :log_directory => nginx_log_dir 72 :log_directory => nginx_log_dir
73 }.merge(params)) 73 }.merge(params))
  74 + log_options node['gitlab']['logging'].to_hash.merge(node['gitlab']['nginx'].to_hash)
74 end 75 end
75 76
76 if node['gitlab']['bootstrap']['enable'] 77 if node['gitlab']['bootstrap']['enable']
files/gitlab-cookbooks/gitlab/recipes/postgresql.rb
  Diff comments   View file @2838abd
@@ -125,10 +125,9 @@ runit_service "postgresql" do @@ -125,10 +125,9 @@ runit_service "postgresql" do
125 down node['gitlab']['postgresql']['ha'] 125 down node['gitlab']['postgresql']['ha']
126 control(['t']) 126 control(['t'])
127 options({ 127 options({
128 - :log_directory => postgresql_log_dir,  
129 - :svlogd_size => node['gitlab']['postgresql']['svlogd_size'],  
130 - :svlogd_num => node['gitlab']['postgresql']['svlogd_num'] 128 + :log_directory => postgresql_log_dir
131 }.merge(params)) 129 }.merge(params))
  130 + log_options node['gitlab']['logging'].to_hash.merge(node['gitlab']['postgresql'].to_hash)
132 end 131 end
133 132
134 if node['gitlab']['bootstrap']['enable'] 133 if node['gitlab']['bootstrap']['enable']
files/gitlab-cookbooks/gitlab/recipes/redis.rb
  Diff comments   View file @2838abd
@@ -55,10 +55,9 @@ end @@ -55,10 +55,9 @@ end
55 runit_service "redis" do 55 runit_service "redis" do
56 down node['gitlab']['redis']['ha'] 56 down node['gitlab']['redis']['ha']
57 options({ 57 options({
58 - :log_directory => redis_log_dir,  
59 - :svlogd_size => node['gitlab']['redis']['svlogd_size'],  
60 - :svlogd_num => node['gitlab']['redis']['svlogd_num'] 58 + :log_directory => redis_log_dir
61 }.merge(params)) 59 }.merge(params))
  60 + log_options node['gitlab']['logging'].to_hash.merge(node['gitlab']['redis'].to_hash)
62 end 61 end
63 62
64 if node['gitlab']['bootstrap']['enable'] 63 if node['gitlab']['bootstrap']['enable']
files/gitlab-cookbooks/gitlab/recipes/sidekiq.rb
  Diff comments   View file @2838abd
@@ -29,6 +29,7 @@ runit_service "sidekiq" do @@ -29,6 +29,7 @@ runit_service "sidekiq" do
29 options({ 29 options({
30 :log_directory => sidekiq_log_dir 30 :log_directory => sidekiq_log_dir
31 }.merge(params)) 31 }.merge(params))
  32 + log_options node['gitlab']['logging'].to_hash.merge(node['gitlab']['sidekiq'].to_hash)
32 end 33 end
33 34
34 if node['gitlab']['bootstrap']['enable'] 35 if node['gitlab']['bootstrap']['enable']
files/gitlab-cookbooks/gitlab/recipes/unicorn.rb
  Diff comments   View file @2838abd
@@ -78,6 +78,7 @@ runit_service "unicorn" do @@ -78,6 +78,7 @@ runit_service "unicorn" do
78 options({ 78 options({
79 :log_directory => unicorn_log_dir 79 :log_directory => unicorn_log_dir
80 }.merge(params)) 80 }.merge(params))
  81 + log_options node['gitlab']['logging'].to_hash.merge(node['gitlab']['unicorn'].to_hash)
81 end 82 end
82 83
83 if node['gitlab']['bootstrap']['enable'] 84 if node['gitlab']['bootstrap']['enable']
files/gitlab-cookbooks/gitlab/templates/default/sv-nginx-log-config.erb 0 → 100644
  Diff comments   View file @2838abd
@@ -0,0 +1,6 @@ @@ -0,0 +1,6 @@
  1 +<%= "s#@svlogd_size" if @svlogd_size %>
  2 +<%= "n#@svlogd_num" if @svlogd_num %>
  3 +<%= "t#@svlogd_timeout" if @svlogd_timeout %>
  4 +<%= "!#@svlogd_filter" if @svlogd_filter %>
  5 +<%= "u#@svlogd_udp" if @svlogd_udp %>
  6 +<%= "p#@svlogd_prefix" if @svlogd_prefix %>
files/gitlab-cookbooks/gitlab/templates/default/sv-postgresql-log-config.erb 0 → 100644
  Diff comments   View file @2838abd
@@ -0,0 +1,6 @@ @@ -0,0 +1,6 @@
  1 +<%= "s#@svlogd_size" if @svlogd_size %>
  2 +<%= "n#@svlogd_num" if @svlogd_num %>
  3 +<%= "t#@svlogd_timeout" if @svlogd_timeout %>
  4 +<%= "!#@svlogd_filter" if @svlogd_filter %>
  5 +<%= "u#@svlogd_udp" if @svlogd_udp %>
  6 +<%= "p#@svlogd_prefix" if @svlogd_prefix %>
files/gitlab-cookbooks/gitlab/templates/default/sv-redis-log-config.erb 0 → 100644
  Diff comments   View file @2838abd
@@ -0,0 +1,6 @@ @@ -0,0 +1,6 @@
  1 +<%= "s#@svlogd_size" if @svlogd_size %>
  2 +<%= "n#@svlogd_num" if @svlogd_num %>
  3 +<%= "t#@svlogd_timeout" if @svlogd_timeout %>
  4 +<%= "!#@svlogd_filter" if @svlogd_filter %>
  5 +<%= "u#@svlogd_udp" if @svlogd_udp %>
  6 +<%= "p#@svlogd_prefix" if @svlogd_prefix %>
files/gitlab-cookbooks/gitlab/templates/default/sv-sidekiq-log-config.erb 0 → 100644
  Diff comments   View file @2838abd
@@ -0,0 +1,6 @@ @@ -0,0 +1,6 @@
  1 +<%= "s#@svlogd_size" if @svlogd_size %>
  2 +<%= "n#@svlogd_num" if @svlogd_num %>
  3 +<%= "t#@svlogd_timeout" if @svlogd_timeout %>
  4 +<%= "!#@svlogd_filter" if @svlogd_filter %>
  5 +<%= "u#@svlogd_udp" if @svlogd_udp %>
  6 +<%= "p#@svlogd_prefix" if @svlogd_prefix %>
files/gitlab-cookbooks/gitlab/templates/default/sv-unicorn-log-config.erb 0 → 100644
  Diff comments   View file @2838abd
@@ -0,0 +1,6 @@ @@ -0,0 +1,6 @@
  1 +<%= "s#@svlogd_size" if @svlogd_size %>
  2 +<%= "n#@svlogd_num" if @svlogd_num %>
  3 +<%= "t#@svlogd_timeout" if @svlogd_timeout %>
  4 +<%= "!#@svlogd_filter" if @svlogd_filter %>
  5 +<%= "u#@svlogd_udp" if @svlogd_udp %>
  6 +<%= "p#@svlogd_prefix" if @svlogd_prefix %>
files/gitlab-cookbooks/runit/definitions/runit_service.rb
  Diff comments   View file @2838abd
@@ -18,7 +18,7 @@ @@ -18,7 +18,7 @@
18 # limitations under the License. 18 # limitations under the License.
19 # 19 #
20 20
21 -define :runit_service, :directory => nil, :only_if => false, :finish_script => false, :control => [], :run_restart => true, :active_directory => nil, :init_script_template => nil, :owner => "root", :group => "root", :template_name => nil, :start_command => "start", :stop_command => "stop", :restart_command => "restart", :status_command => "status", :options => Hash.new, :env => Hash.new, :action => :enable, :down => false do 21 +define :runit_service, :directory => nil, :only_if => false, :finish_script => false, :control => [], :run_restart => true, :active_directory => nil, :init_script_template => nil, :owner => "root", :group => "root", :template_name => nil, :start_command => "start", :stop_command => "stop", :restart_command => "restart", :status_command => "status", :options => Hash.new, :log_options => Hash.new, :env => Hash.new, :action => :enable, :down => false do
22 22
23 include_recipe "runit" 23 include_recipe "runit"
24 24
@@ -77,6 +77,24 @@ define :runit_service, :directory =&gt; nil, :only_if =&gt; false, :finish_script =&gt; f @@ -77,6 +77,24 @@ define :runit_service, :directory =&gt; nil, :only_if =&gt; false, :finish_script =&gt; f
77 end 77 end
78 end 78 end
79 79
  80 + template File.join(params[:options][:log_directory], "config") do
  81 + owner params[:owner]
  82 + group params[:group]
  83 + source "sv-#{params[:template_name]}-log-config.erb"
  84 + cookbook params[:cookbook] if params[:cookbook]
  85 + variables params[:log_options]
  86 + notifies :create, "ruby_block[reload #{params[:name]} svlogd configuration]"
  87 + end
  88 +
  89 + ruby_block "reload #{params[:name]} svlogd configuration" do
  90 + block do
  91 + File.open(File.join(sv_dir_name, "log/supervise/control"), "w") do |control|
  92 + control.print "h"
  93 + end
  94 + end
  95 + action :nothing
  96 + end
  97 +
80 if params[:down] 98 if params[:down]
81 file "#{sv_dir_name}/down" do 99 file "#{sv_dir_name}/down" do
82 mode "0644" 100 mode "0644"