Download as
Browse Code »

Commit 64326fa23d217b1dda0acb3c75b0493e36c2d01a

Authored by Jacob Vosmaer
1 parent 5ff5f662
Exists in master and in 4 other branches 7-1-stable, 7-1-stable-ee, 7-2-stable, 7-2-stable-ee

Add combined svlogd and remote_syslog settings

Showing 4 changed files with 45 additions and 0 deletions   Show diff stats
CHANGELOG
  Diff comments   View file @64326fa
... ... @@ -6,6 +6,7 @@ omnibus-gitlab repository.
6 6 7.1.0
7 7 - Build: explicitly use .forward for sending notifications
8 8 - Fix MySQL build for Ubuntu 14.04
  9 +- Built-in UDP log shipping (Enterprise Edition only)
9 10  
10 11 7.0.0-ee.omnibus.1
11 12 - Fix MySQL build for Ubuntu 14.04
... ...
README.md
  Diff comments   View file @64326fa
... ... @@ -464,6 +464,29 @@ about the files it generates.
464 464  
465 465 You can modify svlogd settings via `/etc/gitlab/gitlab.rb` with the following settings:
466 466  
  467 +### UDP log shipping (GitLab Enterprise Edition only)
  468 +
  469 +You can configure omnibus-gitlab to send syslog-ish log messages via UDP.
  470 +
  471 +```ruby
  472 +logging['udp_log_shipping_host'] = '1.2.3.4' # Your syslog server
  473 +logging['udp_log_shipping_port'] = 1514 # Optional, defaults to 514 (syslog)
  474 +```
  475 +
  476 +Example log messages:
  477 +
  478 +```
  479 +<13>Jun 26 06:33:46 ubuntu1204-test production.log: Started GET "/root/my-project/import" for 127.0.0.1 at 2014-06-26 06:33:46 -0700
  480 +<13>Jun 26 06:33:46 ubuntu1204-test production.log: Processing by ProjectsController#import as HTML
  481 +<13>Jun 26 06:33:46 ubuntu1204-test production.log: Parameters: {"id"=>"root/my-project"}
  482 +<13>Jun 26 06:33:46 ubuntu1204-test production.log: Completed 200 OK in 122ms (Views: 71.9ms | ActiveRecord: 12.2ms)
  483 +<13>Jun 26 06:33:46 ubuntu1204-test gitlab_access.log: 172.16.228.1 - - [26/Jun/2014:06:33:46 -0700] "GET /root/my-project/import HTTP/1.1" 200 5775 "https://172.16.228.169/root/my-project/import" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
  484 +2014-06-26_13:33:46.49866 ubuntu1204-test sidekiq: 2014-06-26T13:33:46Z 18107 TID-7nbj0 Sidekiq::Extensions::DelayedMailer JID-bbfb118dd1db20f6c39f5b50 INFO: start
  485 +
  486 +2014-06-26_13:33:46.52608 ubuntu1204-test sidekiq: 2014-06-26T13:33:46Z 18107 TID-7muoc RepositoryImportWorker JID-57ee926c3655fcfa062338ae INFO: start
  487 +
  488 +```
  489 +
467 490 ```ruby
468 491 # Below are the default values
469 492 logging['svlogd_size'] = 200 * 1024 * 1024 # rotate after 200 MB of log data
... ...
files/gitlab-cookbooks/gitlab/attributes/default.rb
  Diff comments   View file @64326fa
... ... @@ -258,6 +258,8 @@ default[&#39;gitlab&#39;][&#39;logging&#39;][&#39;svlogd_timeout&#39;] = 24 * 60 * 60 # rotate after 24
258 258 default['gitlab']['logging']['svlogd_filter'] = "gzip" # compress logs with gzip
259 259 default['gitlab']['logging']['svlogd_udp'] = nil # transmit log messages via UDP
260 260 default['gitlab']['logging']['svlogd_prefix'] = nil # custom prefix for log messages
  261 +default['gitlab']['logging']['udp_log_shipping_host'] = nil # remote host to ship log messages to via UDP
  262 +default['gitlab']['logging']['udp_log_shipping_port'] = 514 # remote host to ship log messages to via UDP
261 263  
262 264 ###
263 265 # Remote syslog
... ...
files/gitlab-cookbooks/gitlab/libraries/gitlab.rb
  Diff comments   View file @64326fa
... ... @@ -121,6 +121,24 @@ module Gitlab
121 121 Gitlab['gitlab_rails']['satellites_path'] ||= File.join(git_data_dir, "gitlab-satellites")
122 122 end
123 123  
  124 + def parse_udp_log_shipping
  125 + return unless logging['udp_log_shipping_host']
  126 +
  127 + Gitlab['remote_syslog']['enable'] ||= true
  128 + Gitlab['remote_syslog']['destination_host'] ||= logging['udp_log_shipping_host']
  129 +
  130 + if logging['udp_log_shipping_port']
  131 + Gitlab['remote_syslog']['destination_port'] ||= logging['udp_log_shipping_port']
  132 + Gitlab['logging']['svlogd_udp'] ||= "#{logging['udp_log_shipping_host']}:#{logging['udp_log_shipping_port']}"
  133 + else
  134 + Gitlab['logging']['svlogd_udp'] ||= logging['udp_log_shipping_host']
  135 + end
  136 +
  137 + %w{redis nginx sidekiq unicorn postgresql remote-syslog}.each do |runit_sv|
  138 + Gitlab[runit_sv.gsub('-', '_')]['svlogd_prefix'] ||= "#{node['hostname']} #{runit_sv}: "
  139 + end
  140 + end
  141 +
124 142 def generate_hash
125 143 results = { "gitlab" => {} }
126 144 [
... ... @@ -147,6 +165,7 @@ module Gitlab
147 165 generate_secrets(node_name)
148 166 parse_external_url
149 167 parse_git_data_dir
  168 + parse_udp_log_shipping
150 169 # The last step is to convert underscores to hyphens in top-level keys
151 170 generate_hash
152 171 end
... ...