Download as
Browse Code »

Commit 9fd6c3d591fb56e50a9cdbbaac557383e3ef86a7

Authored by Dmitriy Zaporozhets
2 parents 7edbee6c 61c2a2d3
Exists in spb-stable and in 2 other branches 7-0-stable, 7-0-stable-spb

Merge branch 'huge-repo-improve' into 'master' 

Better support for huge repositories

This MR contains next improvements to GitLab:
* You can see repo commits count even if it is > 100k commits
* Prevent 500 for commit page for large repos
* Add notice about huge push over http to unicorn config
* File action in satellites uses default 30 seconds timeout instead of old 10 seconds one
* Show spinner when loading data for network graph
Showing 9 changed files with 37 additions and 10 deletions   Show diff stats
@@ -26,6 +26,11 @@ v 7.0.0 @@ -26,6 +26,11 @@ v 7.0.0
26 - UI improvements for mobile devices 26 - UI improvements for mobile devices
27 - Improve diff rendering performance 27 - Improve diff rendering performance
28 - Drag-n-drop for issues and merge requests between states at milestone page 28 - Drag-n-drop for issues and merge requests between states at milestone page
  29 + - Fix '0 commits' message for huge repositories on project home page
  30 + - Prevent 500 error page when visit commit page from large repo
  31 + - Add notice about huge push over http to unicorn config
  32 + - File action in satellites uses default 30 seconds timeout instead of old 10 seconds one
  33 + - Overall performance improvements
29 34
30 v 6.9.2 35 v 6.9.2
31 - Revert the commit that broke the LDAP user filter 36 - Revert the commit that broke the LDAP user filter
@@ -790,4 +795,4 @@ v 0.8.0 @@ -790,4 +795,4 @@ v 0.8.0
790 - stability 795 - stability
791 - security fixes 796 - security fixes
792 - increased test coverage 797 - increased test coverage
793 - - email notification  
794 \ No newline at end of file 798 \ No newline at end of file
  799 + - email notification
@@ -30,7 +30,7 @@ gem 'omniauth-github' @@ -30,7 +30,7 @@ gem 'omniauth-github'
30 30
31 # Extracting information from a git repository 31 # Extracting information from a git repository
32 # Provide access to Gitlab::Git library 32 # Provide access to Gitlab::Git library
33 -gem "gitlab_git", '~> 5.8' 33 +gem "gitlab_git", '~> 6.0'
34 34
35 # Ruby/Rack Git Smart-HTTP Server Handler 35 # Ruby/Rack Git Smart-HTTP Server Handler
36 gem 'gitlab-grack', '~> 2.0.0.pre', require: 'grack' 36 gem 'gitlab-grack', '~> 2.0.0.pre', require: 'grack'
@@ -175,7 +175,7 @@ GEM @@ -175,7 +175,7 @@ GEM
175 mime-types (~> 1.19) 175 mime-types (~> 1.19)
176 gitlab_emoji (0.0.1.1) 176 gitlab_emoji (0.0.1.1)
177 emoji (~> 1.0.1) 177 emoji (~> 1.0.1)
178 - gitlab_git (5.9.0) 178 + gitlab_git (6.0.0)
179 activesupport (~> 4.0) 179 activesupport (~> 4.0)
180 charlock_holmes (~> 0.6) 180 charlock_holmes (~> 0.6)
181 gitlab-grit (~> 2.6) 181 gitlab-grit (~> 2.6)
@@ -601,7 +601,7 @@ DEPENDENCIES @@ -601,7 +601,7 @@ DEPENDENCIES
601 gitlab-grack (~> 2.0.0.pre) 601 gitlab-grack (~> 2.0.0.pre)
602 gitlab-linguist (~> 3.0.0) 602 gitlab-linguist (~> 3.0.0)
603 gitlab_emoji (~> 0.0.1.1) 603 gitlab_emoji (~> 0.0.1.1)
604 - gitlab_git (~> 5.8) 604 + gitlab_git (~> 6.0)
605 gitlab_meta (= 6.0) 605 gitlab_meta (= 6.0)
606 gitlab_omniauth-ldap (= 1.0.4) 606 gitlab_omniauth-ldap (= 1.0.4)
607 gollum-lib (~> 3.0.0) 607 gollum-lib (~> 3.0.0)
app/controllers/projects/commit_controller.rb
  Diff comments   View file @9fd6c3d
@@ -12,7 +12,12 @@ class Projects::CommitController < Projects::ApplicationController @@ -12,7 +12,12 @@ class Projects::CommitController < Projects::ApplicationController
12 return git_not_found! unless @commit 12 return git_not_found! unless @commit
13 13
14 @line_notes = project.notes.for_commit_id(commit.id).inline 14 @line_notes = project.notes.for_commit_id(commit.id).inline
15 - @branches = project.repository.branch_names_contains(commit.id) 15 +
  16 + @branches = begin
  17 + project.repository.branch_names_contains(commit.id)
  18 + rescue Grit::Git::GitTimeout
  19 + []
  20 + end
16 21
17 begin 22 begin
18 @suppress_diff = true if commit.diff_suppress? && !params[:force_show_diff] 23 @suppress_diff = true if commit.diff_suppress? && !params[:force_show_diff]
app/helpers/application_helper.rb
  Diff comments   View file @9fd6c3d
@@ -226,8 +226,11 @@ module ApplicationHelper @@ -226,8 +226,11 @@ module ApplicationHelper
226 GitHub::Markup.render(file_name, file_content).html_safe 226 GitHub::Markup.render(file_name, file_content).html_safe
227 end 227 end
228 228
229 - def spinner(text = nil)  
230 - content_tag :div, class: 'loading hide' do 229 + def spinner(text = nil, visible = false)
  230 + css_class = "loading"
  231 + css_class << " hide" unless visible
  232 +
  233 + content_tag :div, class: css_class do
231 content_tag(:i, nil, class: 'icon-spinner icon-spin') + text 234 content_tag(:i, nil, class: 'icon-spinner icon-spin') + text
232 end 235 end
233 end 236 end
app/models/repository.rb
  Diff comments   View file @9fd6c3d
@@ -106,7 +106,7 @@ class Repository @@ -106,7 +106,7 @@ class Repository
106 def commit_count 106 def commit_count
107 Rails.cache.fetch(cache_key(:commit_count)) do 107 Rails.cache.fetch(cache_key(:commit_count)) do
108 begin 108 begin
109 - raw_repository.raw.commit_count(self.root_ref) 109 + raw_repository.commit_count(self.root_ref)
110 rescue 110 rescue
111 0 111 0
112 end 112 end
app/views/projects/network/show.html.haml
  Diff comments   View file @9fd6c3d
@@ -3,7 +3,7 @@ @@ -3,7 +3,7 @@
3 .tip 3 .tip
4 You can move around the graph by using the arrow keys. 4 You can move around the graph by using the arrow keys.
5 .network-graph 5 .network-graph
6 - = spinner 6 + = spinner nil, true
7 7
8 :javascript 8 :javascript
9 new Network({ 9 new Network({
config/unicorn.rb.example
  Diff comments   View file @9fd6c3d
@@ -34,6 +34,20 @@ listen &quot;/home/git/gitlab/tmp/sockets/gitlab.socket&quot;, :backlog =&gt; 64 @@ -34,6 +34,20 @@ listen &quot;/home/git/gitlab/tmp/sockets/gitlab.socket&quot;, :backlog =&gt; 64
34 listen "127.0.0.1:8080", :tcp_nopush => true 34 listen "127.0.0.1:8080", :tcp_nopush => true
35 35
36 # nuke workers after 30 seconds instead of 60 seconds (the default) 36 # nuke workers after 30 seconds instead of 60 seconds (the default)
  37 +#
  38 +# NOTICE: git push over http depends on this value.
  39 +# If you want be able to push huge amount of data to git repository over http
  40 +# you will have to increase this value too.
  41 +#
  42 +# Example of output if you try to push 1GB repo to GitLab over http.
  43 +# -> git push http://gitlab.... master
  44 +#
  45 +# error: RPC failed; result=18, HTTP code = 200
  46 +# fatal: The remote end hung up unexpectedly
  47 +# fatal: The remote end hung up unexpectedly
  48 +#
  49 +# For more information see http://stackoverflow.com/a/21682112/752049
  50 +#
37 timeout 30 51 timeout 30
38 52
39 # feel free to point this anywhere accessible on the filesystem 53 # feel free to point this anywhere accessible on the filesystem
lib/gitlab/satellite/files/file_action.rb
  Diff comments   View file @9fd6c3d
@@ -4,7 +4,7 @@ module Gitlab @@ -4,7 +4,7 @@ module Gitlab
4 attr_accessor :file_path, :ref 4 attr_accessor :file_path, :ref
5 5
6 def initialize(user, project, ref, file_path) 6 def initialize(user, project, ref, file_path)
7 - super user, project, git_timeout: 10.seconds 7 + super user, project
8 @file_path = file_path 8 @file_path = file_path
9 @ref = ref 9 @ref = ref
10 end 10 end