Fix Wiki migration task and add more test coverage.

There was an error in the #extract_attributes_from_page method that caused an exception when checking to see if a page named "Home" already exists. The check occurs to handle the renaming of the main index page to "Home" to match the Gollum standard. If there is already a page called "Home" then the migrator will leave that page and create the Index page as usual. Users will need to manually rename their old "Home" page to something else then rename their "Index" page to "Home". Fortunately, I would be highly surprised if this case ever comes up. I also added more verbosity to the logging so if an error does occur, it will be easier to track down which Wiki page is causing the problem.

Fix Wiki migration task and add more test coverage.
There was an error in the #extract_attributes_from_page method that caused an exception when checking to see if a page named "Home" already exists. The check occurs to handle the renaming of the main index page to "Home" to match the Gollum standard. If there is already a page called "Home" then the migrator will leave that page and create the Index page as usual. Users will need to manually rename their old "Home" page to something else then rename their "Index" page to "Home". Fortunately, I would be highly surprised if this case ever comes up. I also added more verbosity to the logging so if an error does occur, it will be easier to track down which Wiki page is causing the problem.
Dan Knox
1 parent c367cc5c
Showing 3 changed files with 154 additions and 6 deletions Show diff stats
lib/tasks/gitlab/migrate_wiki.rake
lib/wiki_to_gollum_migrator.rb
spec/lib/wiki_to_gollum_migrator_spec.rb
@@ -16,5 +16,27 @@ namespace :gitlab do
       wiki_migrator = WikiToGollumMigrator.new
       wiki_migrator.migrate!
     end
+
+    # This task will destroy all of the Wiki repos
+    # that the Wiki migration task created. Run this
+    # to clean up your environment if you experienced
+    # problems during the original migration. After
+    # executing this task, you can attempt the original
+    # migration again.
+    #
+    # Notes:
+    #  * This will not affect Wikis that have been created
+    #    as Gollum Wikis only. It will only remove the wikis
+    #    for the repositories that have old Wiki data in the
+    #    dataabase.
+    #  * If you have any repositories already named
+    #    namespace/project.wiki that you do not wish
+    #    to be removed you may want to perform a manual
+    #    cleanup instead.
+    desc "GITLAB | Remove the Wiki repositories created by the `gitlab:wiki:migrate` task."
+    task :rollback => :environment do
+      wiki_migrator = WikiToGollumMigrator.new
+      wiki_migrator.rollback!
+    end
   end
 end
@@ -19,12 +19,30 @@ class WikiToGollumMigrator
     end
   end
+  def rollback!
+    log "\nBeginning Wiki Migration Rollback..."
+    projects.each do |project|
+      destroy_gollum_repo project
+    end
+    log "\nWiki Rollback Complete."
+  end
+
   private
   def create_gollum_repo(project)
     GollumWiki.new(project, nil).wiki
   end
+  def destroy_gollum_repo(project)
+    log "  Removing Wiki repo for project: #{project.path_with_namespace}"
+    path = GollumWiki.new(project, nil).path_with_namespace
+    if Gitlab::Shell.new.remove_repository(path)
+      log "  Wiki destroyed successfully. " + "[OK}".green
+    else
+      log "  Problem destroying wiki. Please remove it manually. " + "[FAILED]".red
+    end
+  end
+
   def create_pages(project, wiki)
     pages = project.wikis.group(:slug).all
@@ -45,8 +63,9 @@ class WikiToGollumMigrator
     wiki = GollumWiki.new(project, page.user)
     wiki_page = WikiPage.new(wiki)
-    attributes = extract_attributes_from_page(first_rev)
+    attributes = extract_attributes_from_page(first_rev, project)
+    log "  Creating page '#{first_rev.title}'..."
     if wiki_page.create(attributes)
       log "  Created page '#{wiki_page.title}' " + "[OK]".green
@@ -59,15 +78,15 @@ class WikiToGollumMigrator
   end
   def create_revisions(project, page, revisions)
+    log "    Creating revisions..."
     revisions.each do |revision|
-      log "    Creating revisions..."
       # Reinitialize a new GollumWiki instance for each page
       # and revision created so the correct User is shown in
       # the commit message.
       wiki = GollumWiki.new(project, revision.user)
       wiki_page = wiki.find_page(page.slug)
-      attributes = extract_attributes_from_page(revision)
+      attributes = extract_attributes_from_page(revision, project)
       content = attributes[:content]
@@ -79,13 +98,15 @@ class WikiToGollumMigrator
     end
   end
-  def extract_attributes_from_page(page)
+  def extract_attributes_from_page(page, project)
     attributes = page.attributes
                      .with_indifferent_access
                      .slice(:title, :content)
+    slug = page.slug
+
     # Change 'index' pages to 'home' pages to match Gollum standards
-    if attributes[:title].downcase == "index"
+    if slug.downcase == "index"
       attributes[:title] = "home" unless home_already_exists?(project)
     end
@@ -93,7 +114,7 @@ class WikiToGollumMigrator
   end
   def home_already_exists?(project)
-    project.wikis.where(title: 'home').any? || project.wikis.where(title: 'Home').any?
+    project.wikis.where(slug: 'home').any? || project.wikis.where(slug: 'Home').any?
   end
   def log(message)
@@ -108,6 +108,111 @@ describe WikiToGollumMigrator do
         end
       end
     end
+
+    context "wikis with pages that have titles that do not match the slugs" do
+      before do
+        project = @projects.last
+        @page = project.wikis.new(title: "test page", content: "Invalid Page")
+        @page.slug = "totally-incorrect-slug"
+        @page.user = project.owner
+        @page.save!
+
+        create_revision(@page)
+
+        subject.rollback!
+        subject.migrate!
+      end
+
+      it "has a page with a title differing the slug" do
+        @page.slug.should_not == @page.title.parameterize
+      end
+
+      it "creates a new revision for each old revision of the page" do
+        @projects.each do |project|
+          wiki = GollumWiki.new(project, nil)
+          wiki.pages.each do |page|
+            page.versions.count.should == 2
+          end
+        end
+      end
+    end
+
+    context "changing wiki title from index to home" do
+      before do
+        @project = @projects.last
+        @page = @project.wikis.new(title: "Index", content: "Home Page")
+        @page.slug = "index"
+        @page.user = @project.owner
+        @page.save!
+
+        create_revision(@page)
+
+        subject.rollback!
+      end
+
+      it "creates a page called Home" do
+        subject.migrate!
+        wiki = GollumWiki.new(@project, nil)
+        page = wiki.find_page("home")
+        page.should be_present
+      end
+
+      context "when a page called Home already exists" do
+        before do
+        @index_page = @project.wikis.new(title: "Index", content: "Index Page")
+        @index_page.slug = "index"
+        @index_page.user = @project.owner
+        @index_page.save!
+
+        create_revision(@index_page)
+
+        @home_page = @project.wikis.new(title: "Home", content: "Home Page")
+        @home_page.slug = "home"
+        @home_page.user = @project.owner
+        @home_page.save!
+
+        create_revision(@home_page)
+        subject.migrate!
+        end
+
+        it "creates the index page" do
+          wiki = GollumWiki.new(@project, nil)
+          page = wiki.find_page("index")
+          page.should be_present
+        end
+
+        it "creates the home page" do
+          wiki = GollumWiki.new(@project, nil)
+          page = wiki.find_page("home")
+          page.should be_present
+        end
+      end
+    end
+  end
+
+  context "#rollback!" do
+    before do
+      Gitlab::Shell.any_instance.stub(:add_repository) do |path|
+        create_temp_repo("#{@repo_path}/#{path}.git")
+      end
+
+      Gitlab::Shell.any_instance.stub(:remove_repository) do |path|
+        FileUtils.rm_rf "#{@repo_path}/#{path}.git"
+      end
+
+      subject.stub(:log).as_null_object
+
+      subject.migrate!
+      subject.rollback!
+    end
+
+    it "destroys all of the wiki repositories that were created during migrate!" do
+      @projects.each do |project|
+        wiki_path = project.path_with_namespace + ".wiki.git"
+        full_path = @repo_path + "/" + wiki_path
+        File.exist?(full_path).should be_false
+      end
+    end
   end
	@@ -16,5 +16,27 @@ namespace :gitlab do		@@ -16,5 +16,27 @@ namespace :gitlab do
16	wiki_migrator = WikiToGollumMigrator.new	16	wiki_migrator = WikiToGollumMigrator.new
17	wiki_migrator.migrate!	17	wiki_migrator.migrate!
18	end	18	end
		19	+
		20	+ # This task will destroy all of the Wiki repos
		21	+ # that the Wiki migration task created. Run this
		22	+ # to clean up your environment if you experienced
		23	+ # problems during the original migration. After
		24	+ # executing this task, you can attempt the original
		25	+ # migration again.
		26	+ #
		27	+ # Notes:
		28	+ # * This will not affect Wikis that have been created
		29	+ # as Gollum Wikis only. It will only remove the wikis
		30	+ # for the repositories that have old Wiki data in the
		31	+ # dataabase.
		32	+ # * If you have any repositories already named
		33	+ # namespace/project.wiki that you do not wish
		34	+ # to be removed you may want to perform a manual
		35	+ # cleanup instead.
		36	+ desc "GITLAB \| Remove the Wiki repositories created by the `gitlab:wiki:migrate` task."
		37	+ task :rollback => :environment do
		38	+ wiki_migrator = WikiToGollumMigrator.new
		39	+ wiki_migrator.rollback!
		40	+ end
19	end	41	end
20	end	42	end
	@@ -108,6 +108,111 @@ describe WikiToGollumMigrator do		@@ -108,6 +108,111 @@ describe WikiToGollumMigrator do
108	end	108	end
109	end	109	end
110	end	110	end
		111	+
		112	+ context "wikis with pages that have titles that do not match the slugs" do
		113	+ before do
		114	+ project = @projects.last
		115	+ @page = project.wikis.new(title: "test page", content: "Invalid Page")
		116	+ @page.slug = "totally-incorrect-slug"
		117	+ @page.user = project.owner
		118	+ @page.save!
		119	+
		120	+ create_revision(@page)
		121	+
		122	+ subject.rollback!
		123	+ subject.migrate!
		124	+ end
		125	+
		126	+ it "has a page with a title differing the slug" do
		127	+ @page.slug.should_not == @page.title.parameterize
		128	+ end
		129	+
		130	+ it "creates a new revision for each old revision of the page" do
		131	+ @projects.each do \|project\|
		132	+ wiki = GollumWiki.new(project, nil)
		133	+ wiki.pages.each do \|page\|
		134	+ page.versions.count.should == 2
		135	+ end
		136	+ end
		137	+ end
		138	+ end
		139	+
		140	+ context "changing wiki title from index to home" do
		141	+ before do
		142	+ @project = @projects.last
		143	+ @page = @project.wikis.new(title: "Index", content: "Home Page")
		144	+ @page.slug = "index"
		145	+ @page.user = @project.owner
		146	+ @page.save!
		147	+
		148	+ create_revision(@page)
		149	+
		150	+ subject.rollback!
		151	+ end
		152	+
		153	+ it "creates a page called Home" do
		154	+ subject.migrate!
		155	+ wiki = GollumWiki.new(@project, nil)
		156	+ page = wiki.find_page("home")
		157	+ page.should be_present
		158	+ end
		159	+
		160	+ context "when a page called Home already exists" do
		161	+ before do
		162	+ @index_page = @project.wikis.new(title: "Index", content: "Index Page")
		163	+ @index_page.slug = "index"
		164	+ @index_page.user = @project.owner
		165	+ @index_page.save!
		166	+
		167	+ create_revision(@index_page)
		168	+
		169	+ @home_page = @project.wikis.new(title: "Home", content: "Home Page")
		170	+ @home_page.slug = "home"
		171	+ @home_page.user = @project.owner
		172	+ @home_page.save!
		173	+
		174	+ create_revision(@home_page)
		175	+ subject.migrate!
		176	+ end
		177	+
		178	+ it "creates the index page" do
		179	+ wiki = GollumWiki.new(@project, nil)
		180	+ page = wiki.find_page("index")
		181	+ page.should be_present
		182	+ end
		183	+
		184	+ it "creates the home page" do
		185	+ wiki = GollumWiki.new(@project, nil)
		186	+ page = wiki.find_page("home")
		187	+ page.should be_present
		188	+ end
		189	+ end
		190	+ end
		191	+ end
		192	+
		193	+ context "#rollback!" do
		194	+ before do
		195	+ Gitlab::Shell.any_instance.stub(:add_repository) do \|path\|
		196	+ create_temp_repo("#{@repo_path}/#{path}.git")
		197	+ end
		198	+
		199	+ Gitlab::Shell.any_instance.stub(:remove_repository) do \|path\|
		200	+ FileUtils.rm_rf "#{@repo_path}/#{path}.git"
		201	+ end
		202	+
		203	+ subject.stub(:log).as_null_object
		204	+
		205	+ subject.migrate!
		206	+ subject.rollback!
		207	+ end
		208	+
		209	+ it "destroys all of the wiki repositories that were created during migrate!" do
		210	+ @projects.each do \|project\|
		211	+ wiki_path = project.path_with_namespace + ".wiki.git"
		212	+ full_path = @repo_path + "/" + wiki_path
		213	+ File.exist?(full_path).should be_false
		214	+ end
		215	+ end
111	end	216	end
112		217
113		218