Adding multitenancy support

Developed with Caio SBA <caio@colivre.coop.br> Available only for PostgreSQL by now See INSTALL.multitenancy for details (ActionItem1845)

Adding multitenancy support
Developed with Caio SBA <caio@colivre.coop.br> Available only for PostgreSQL by now See INSTALL.multitenancy for details (ActionItem1845)
Daniel Cunha · Daniela Feitosa
1 parent 909c8e7f
Showing 32 changed files with 518 additions and 34 deletions Show diff stats
INSTALL.multitenancy
app/controllers/application.rb
app/controllers/box_organizer_controller.rb
app/helpers/sweeper_helper.rb
app/models/block.rb
app/models/image.rb
app/models/thumbnail.rb
app/models/uploaded_file.rb
app/sweepers/article_sweeper.rb
app/sweepers/friendship_sweeper.rb
app/sweepers/profile_sweeper.rb
app/sweepers/role_assignment_sweeper.rb
app/views/shared/block.rhtml
config/database.yml.multitenancy
config/initializers/postgresql_attachment_fu.rb
lib/acts_as_searchable.rb
lib/noosfero/core_ext.rb
lib/noosfero/core_ext/active_record.rb
lib/noosfero/multi_tenancy.rb
lib/postgresql_attachment_fu.rb
@@ -0,0 +1,139 @@
+== Multitenancy support
+
+Multitenancy refers to a principle in software architecture where a
+single instance of the software runs on a server, serving multiple
+client organizations (tenants). Multitenancy is contrasted with a
+multi-instance architecture where separate software instances (or
+hardware systems) are set up for different client organizations. With
+a multitenant architecture, a software application is designed to
+virtually partition its data and configuration, and each client
+organization works with a customized virtual application instance.
+
+Today this feature is available only for PostgreSQL databases.
+
+This document assumes that you have a fully PostgresSQL default Noosfero
+installation as explained at the INSTALL file.
+
+== Separated data
+
+The items below are separated for each hosted environment:
+
+* Uploaded files
+* Database
+* Ferret index
+* ActiveRecord#cache_key
+* Feed updater
+* Delayed Job Workers
+
+== Database configuration file
+
+The file config/database.yml must follow a structure in order to
+achieve multitenancy support. In this example, we will set 3
+different environments: env1, env2 and env3.
+
+Each "hosted" environment must have an entry like this:
+
+env1_production:
+  adapter: postgresql
+  encoding: unicode
+  database: noosfero
+  schema_search_path: public
+  username: noosfero
+  domains:
+    - env1.com
+    - env1.org
+
+env2_production:
+  adapter: postgresql
+  encoding: unicode
+  database: noosfero
+  schema_search_path: env2
+  username: noosfero
+  domains:
+    - env2.com
+    - env2.org
+
+env3_production:
+  adapter: postgresql
+  encoding: unicode
+  database: noosfero
+  schema_search_path: env3
+  username: noosfero
+  domains:
+    - env3.com
+    - env3.net
+
+The "hosted" environments define, besides the schema_search_path, a
+list of domains that, when accessed, tells which database the
+application should use. Also, the environment name must end with
+'_hosting', where 'hosting' is the name of the hosting environment.
+
+You must also tell the application which is the default environment.
+
+production:
+  env1_production
+
+On the example above there are only three hosted environments, but it
+can be more than three. The schemas 'env2' and 'env3' must already
+exist in the same database of the hosting environment. As postgres
+user, you can create them typing:
+
+$ psql database_name -c "CREATE SCHEMA env2 AUTHORIZATION database_user"
+$ psql database_name -c "CREATE SCHEMA env3 AUTHORIZATION database_user"
+
+Replace database_name and database_user above with your stuff.
+
+So, yet on this same example, when a user accesses http://env2.com or
+http://env2.org, the Noosfero application running on production will
+turn the database schema to 'env2'. When the access is from domains
+http://env3.com or http://env3.net, the schema to be loaded will be
+'env3'.
+
+There is an example of this file in config/database.yml.multitenancy
+
+== Preparing the database
+
+Now create the environments:
+
+$ RAILS_ENV=production rake multitenancy:create
+
+This command above will create the hosted environment files equal to
+their hosting environment, here called 'production'.
+
+Run db:schema:load for each other environment:
+
+$ RAILS_ENV=env2_production rake db:schema:load
+$ RAILS_ENV=env3_production rake db:schema:load
+
+Then run the migrations for the hosting environment, and it will
+run for each of its hosted environments:
+
+RAILS_ENV=production rake db:migrate
+
+== Start Noosfero
+
+Run Noosfero init file as root:
+
+# invoke-rc.d noosfero start
+
+== Ferret
+
+It's necessary to run only one instance of ferret_server. Don't worry
+about this, Noosfero initializer had already done this for you.
+
+Build or rebuild the Ferret index by running the following task just
+for your hosting environment, do this as noosfero user:
+
+$ RAILS_ENV=production rake multitenancy:reindex
+
+== Feed updater & Delayed job
+
+Just for your information, a daemon of feed-updater and delayed_job
+must be running for each environment. Noosfero initializer do this,
+relax.
+
+== Uploaded files
+
+When running with PostgreSQL, Noosfero uploads stuff to a folder named
+the same way as the running schema. Inside the upload folder root, for
+example, will be public/image_uploads/env2 and public/image_uploads/env3.
@@ -2,6 +2,8 @@
 # available in all controllers.
 class ApplicationController < ActionController::Base
+  before_filter :change_pg_schema
+
   include ApplicationHelper
   layout :get_layout
   def get_layout
@@ -96,6 +98,12 @@ class ApplicationController &lt; ActionController::Base
   helper_method :current_person, :current_person
+  def change_pg_schema
+    if Noosfero::MultiTenancy.on? and ActiveRecord::Base.postgresql?
+      Noosfero::MultiTenancy.db_by_host = request.host
+    end
+  end
+
   protected
   def user
@@ -82,7 +82,7 @@ class BoxOrganizerController &lt; ApplicationController
   def save
     @block = boxes_holder.blocks.find(params[:id])
     @block.update_attributes(params[:block])
-    expire_timeout_fragment(@block.cache_keys)
+    expire_timeout_fragment(@block.cache_key)
     redirect_to :action => 'index'
   end
@@ -93,7 +93,7 @@ class BoxOrganizerController &lt; ApplicationController
   def remove
     @block = Block.find(params[:id])
     if @block.destroy
-      expire_timeout_fragment(@block.cache_keys)
+      expire_timeout_fragment(@block.cache_key)
       redirect_to :action => 'index'
     else
       session[:notice] = _('Failed to remove block')
@@ -22,7 +22,7 @@ module SweeperHelper
     # friends blocks
     blocks = profile.blocks.select{|b| b.kind_of?(FriendsBlock)}
-    blocks.map(&:cache_keys).each{|ck|expire_timeout_fragment(ck)}
+    blocks.map(&:cache_key).each{|ck|expire_timeout_fragment(ck)}
   end
   def expire_communities(profile)
@@ -34,13 +34,13 @@ module SweeperHelper
     # communities block
     blocks = profile.blocks.select{|b| b.kind_of?(CommunitiesBlock)}
-    blocks.map(&:cache_keys).each{|ck|expire_timeout_fragment(ck)}
+    blocks.map(&:cache_key).each{|ck|expire_timeout_fragment(ck)}
   end
   def expire_enterprises(profile)
     # enterprises and favorite enterprises blocks
     blocks = profile.blocks.select {|b| [EnterprisesBlock, FavoriteEnterprisesBlock].any?{|klass| b.kind_of?(klass)} }
-    blocks.map(&:cache_keys).each{|ck|expire_timeout_fragment(ck)}
+    blocks.map(&:cache_key).each{|ck|expire_timeout_fragment(ck)}
   end
   def expire_profile_index(profile)
@@ -119,10 +119,6 @@ class Block &lt; ActiveRecord::Base
     true
   end
-  def cache_keys
-    "block-id-#{id}"
-  end
-
   def timeout
     4.hours
   end
@@ -20,4 +20,6 @@ class Image &lt; ActiveRecord::Base
   delay_attachment_fu_thumbnails
+  postgresql_attachment_fu
+
 end
@@ -2,4 +2,6 @@ class Thumbnail &lt; ActiveRecord::Base
   has_attachment :storage => :file_system,
     :content_type => :image, :max_size => 5.megabytes
   validates_as_attachment
+
+  postgresql_attachment_fu
 end
@@ -52,6 +52,8 @@ class UploadedFile &lt; Article
   delay_attachment_fu_thumbnails
+  postgresql_attachment_fu
+
   def self.icon_name(article = nil)
     if article
       article.image? ? article.public_filename(:icon) : (article.mime_type ? article.mime_type.gsub(/[\/+.]/, '-') : 'upload-file')
@@ -21,7 +21,7 @@ protected
     blocks = article.profile.blocks
     blocks += article.profile.environment.blocks if article.profile.environment
     blocks = blocks.select{|b|[RecentDocumentsBlock, BlogArchivesBlock].any?{|c| b.kind_of?(c)}}
-    blocks.map(&:cache_keys).each{|ck|expire_timeout_fragment(ck)}
+    blocks.map(&:cache_key).each{|ck|expire_timeout_fragment(ck)}
     env = article.profile.environment
     if env && (env.portal_community == article.profile)
       expire_fragment(env.portal_news_cache_key)
@@ -35,7 +35,7 @@ protected
     end
     blocks = profile.blocks.select{|b| b.kind_of?(FriendsBlock)}
-    blocks.map(&:cache_keys).each{|ck|expire_timeout_fragment(ck)}
+    blocks.map(&:cache_key).each{|ck|expire_timeout_fragment(ck)}
   end
 end
@@ -23,12 +23,12 @@ protected
     expire_profile_index(profile) if profile.person?
     profile.blocks.each do |block|
-      expire_timeout_fragment(block.cache_keys)
+      expire_timeout_fragment(block.cache_key)
     end
   end
   def expire_statistics_block_cache(profile)
     blocks = profile.environment.blocks.select { |b| b.kind_of?(EnvironmentStatisticsBlock) }
-    blocks.map(&:cache_keys).each{|ck|expire_timeout_fragment(ck)}
+    blocks.map(&:cache_key).each{|ck|expire_timeout_fragment(ck)}
   end
 end
@@ -25,7 +25,7 @@ protected
     profile.blocks_to_expire_cache.each { |block|
       blocks = profile.blocks.select{|b| b.kind_of?(block)}
-      blocks.map(&:cache_keys).each{|ck|expire_timeout_fragment(ck)}
+      blocks.map(&:cache_key).each{|ck|expire_timeout_fragment(ck)}
     }
   end
 <% if block.cacheable? && use_cache %>
-  <% cache_timeout(block.cache_keys, block.timeout.from_now) do %>
+  <% cache_timeout(block.cache_key, block.timeout.from_now) do %>
     <%= display_block_content(block, main_content) %>
   <% end %>
 <% else %>
@@ -0,0 +1,33 @@
+# Refer to INSTALL.multitenancy for more information on Multitenancy support
+env1_production:
+  adapter: postgresql
+  encoding: unicode
+  database: noosfero
+  schema_search_path: public
+  username: noosfero
+  domains:
+    - env1.com
+    - env1.org
+
+env2_production:
+  adapter: postgresql
+  encoding: unicode
+  database: noosfero
+  schema_search_path: env2
+  username: noosfero
+  domains:
+    - env2.com
+    - env2.org
+
+env3_production:
+  adapter: postgresql
+  encoding: unicode
+  database: noosfero
+  schema_search_path: env3
+  username: noosfero
+  domains:
+    - env3.com
+    - env3.net
+
+production:
+  env1_production
@@ -0,0 +1 @@
+require 'postgresql_attachment_fu'
@@ -2,11 +2,28 @@ module ActsAsSearchable
   module ClassMethods
     def acts_as_searchable(options = {})
+      if ActiveRecord::Base.connection.adapter_name == 'PostgreSQL'
+        options[:additional_fields] ||= {}
+        options[:additional_fields] = Hash[*options[:additional_fields].collect{ |v| [v, {}] }.flatten] if options[:additional_fields].is_a?(Array)
+        options[:additional_fields].merge!(:schema_name => { :index => :untokenized })
+      end
       acts_as_ferret({ :remote => true }.merge(options))
       extend FindByContents
+      send :include, InstanceMethods
+    end
+
+    module InstanceMethods
+      def schema_name
+        ActiveRecord::Base.connection.schema_search_path
+      end
     end
     module FindByContents
+
+      def schema_name
+        ActiveRecord::Base.connection.schema_search_path
+      end
+
       def find_by_contents(query, ferret_options = {}, db_options = {})
         pg_options = {}
         if ferret_options[:page]
@@ -18,8 +35,9 @@ module ActsAsSearchable
         ferret_options[:limit] = :all
+        ferret_query = ActiveRecord::Base.connection.adapter_name == 'PostgreSQL' ? "+schema_name:\"#{schema_name}\" AND #{query}" : query
         # FIXME this is a HORRIBLE HACK
-        ids = find_ids_with_ferret(query, ferret_options)[1][0..8000].map{|r|r[:id].to_i}
+        ids = find_ids_with_ferret(ferret_query, ferret_options)[1][0..8000].map{|r|r[:id].to_i}
         if ids.empty?
           ids << -1
 require 'noosfero/core_ext/string'
 require 'noosfero/core_ext/integer'
 require 'noosfero/core_ext/object'
+require 'noosfero/core_ext/active_record'
@@ -0,0 +1,14 @@
+class ActiveRecord::Base
+
+  def self.postgresql?
+    ActiveRecord::Base.connection.adapter_name == 'PostgreSQL'
+  end
+
+  alias :meta_cache_key :cache_key
+  def cache_key
+    key = [Noosfero::VERSION, meta_cache_key]
+    key.unshift(ActiveRecord::Base.connection.schema_search_path) if ActiveRecord::Base.postgresql?
+    key.join('/')
+  end
+
+end
@@ -0,0 +1,30 @@
+module Noosfero
+  class MultiTenancy
+
+    def self.mapping
+      @mapping ||= self.load_map
+    end
+
+    def self.on?
+      !self.mapping.blank?
+    end
+
+    def self.db_by_host=(host)
+      ActiveRecord::Base.connection.schema_search_path = self.mapping[host]
+    end
+
+    private
+
+    def self.load_map
+      db_file = File.join(RAILS_ROOT, 'config', 'database.yml')
+      db_config = YAML.load_file(db_file)
+      map = { }
+      db_config.each do |env, attr|
+        next unless env.match(/_#{RAILS_ENV}$/) and attr['adapter'] =~ /^postgresql$/i
+        attr['domains'].each { |d| map[d] = attr['schema_search_path'] }
+      end
+      map
+    end
+
+  end
+end
@@ -0,0 +1,19 @@
+module PostgresqlAttachmentFu
+
+  module ClassMethods
+    def postgresql_attachment_fu
+      send :include, InstanceMethods
+    end
+  end
+
+  module InstanceMethods
+    def full_filename(thumbnail = nil)
+      file_system_path = (thumbnail ? thumbnail_class : self).attachment_options[:path_prefix].to_s
+      file_system_path = File.join(file_system_path, ActiveRecord::Base.connection.schema_search_path) if ActiveRecord::Base.postgresql?
+      File.join(RAILS_ROOT, file_system_path, *partitioned_path(thumbnail_name_for(thumbnail)))
+    end
+  end
+
+end
+
+ActiveRecord::Base.send(:extend, PostgresqlAttachmentFu::ClassMethods)
@@ -0,0 +1,47 @@
+namespace :multitenancy do
+
+  task :create do
+    db_envs = ActiveRecord::Base.configurations.keys.select{ |k| k.match(/_development$|_production$|_test$/) }
+    cd File.join(RAILS_ROOT, 'config', 'environments'), :verbose => true
+    file_envs = Dir.glob "{*_development.rb,*_prodution.rb,*_test.rb}"
+    (db_envs.map{ |e| e + '.rb' } - file_envs).each { |env| ln_s env.split('_').last, env }
+  end
+
+  task :remove do
+    db_envs = ActiveRecord::Base.configurations.keys.select{ |k| k.match(/_development$|_production$|_test$/) }
+    cd File.join(RAILS_ROOT, 'config', 'environments'), :verbose => true
+    file_envs = Dir.glob "{*_development.rb,*_prodution.rb,*_test.rb}"
+    (file_envs - db_envs.map{ |e| e + '.rb' }).each { |env| safe_unlink env }
+  end
+
+  task :reindex => :environment do
+    envs = ActiveRecord::Base.configurations.keys.select{ |k| k.match(/_#{RAILS_ENV}$/) }
+    models = [ Profile, Article, Product ]
+    envs.each do |e|
+      puts "Rebuilding Index for #{e}" if Rake.application.options.trace
+      ActiveRecord::Base.connection.schema_search_path = ActiveRecord::Base.configurations[e]['schema_search_path']
+      models.each do |m|
+        if e == envs[0]
+          m.rebuild_index
+          puts "Rebuilt index for #{m}" if Rake.application.options.trace
+        end
+        m.paginated_each(:per_page => 50) { |i| i.ferret_update }
+        puts "Reindexed all instances of #{m}" if Rake.application.options.trace
+      end
+    end
+  end
+
+end
+
+namespace :db do
+
+  task :migrate_other_environments => :environment do
+    envs = ActiveRecord::Base.configurations.keys.select{ |k| k.match(/_#{RAILS_ENV}$/) }
+    envs.each do |e|
+      puts "*** Migrating #{e}" if Rake.application.options.trace
+      system "rake db:migrate RAILS_ENV=#{e}"
+    end
+  end
+  task :migrate => :migrate_other_environments
+
+end
@@ -7,6 +7,7 @@
 # etc. The actual feed update logic is in FeedUpdater.
 require 'daemons'
+require 'optparse'
 NOOSFERO_ROOT = File.expand_path(File.dirname(__FILE__) + '/../')
@@ -15,11 +16,16 @@ options = {
   :dir        => File.dirname(__FILE__) + '/../tmp/pids',
   :multiple   => false,
   :backtrace  => true,
-  :monitor    => true,
+  :monitor    => false
 }
-Daemons.run_proc('feed-updater', options) do
+OptionParser.new do |opts|
+  opts.on("-i", "--identifier=i", "Id") do |i|
+    options[:identifier] = i
+  end
+end.parse!(ARGV)
+
+Daemons.run_proc("feed-updater.#{options[:identifier]}", options) do
   require NOOSFERO_ROOT + '/config/environment'
   FeedUpdater.new.start
 end
-
@@ -23,8 +23,7 @@ do_start() {
   clear_cache
   ./script/ferret_server -e $RAILS_ENV start
-  ./script/feed-updater start
-  ./script/delayed_job start
+  environments_loop
   mongrel_rails cluster::start
 }
@@ -35,6 +34,20 @@ do_stop() {
   ./script/ferret_server -e $RAILS_ENV stop
 }
+environments_loop() {
+  environments=$(find ./config/environments -name *_$RAILS_ENV.rb)
+  if [ "$environments" ]; then
+    for environment in $environments; do
+      env=$(basename $environment | cut -d. -f1)
+      RAILS_ENV=$env ./script/delayed_job -i $env start
+      RAILS_ENV=$env ./script/feed-updater start -i $env
+    done
+  else
+    ./script/delayed_job start
+    ./script/feed-updater start
+  fi
+}
+
 case "$ACTION" in
   start|stop)
     do_$ACTION
@@ -58,4 +71,3 @@ case &quot;$ACTION&quot; in
     exit 1
     ;;
 esac
-
@@ -62,7 +62,7 @@ class ApplicationControllerTest &lt; Test::Unit::TestCase
   def test_local_files_reference
     assert_local_files_reference
   end
-  
+
   def test_valid_xhtml
     assert_valid_xhtml
   end
@@ -384,7 +384,7 @@ class ApplicationControllerTest &lt; Test::Unit::TestCase
     uses_host 'other.environment'
     get :index
     assert_tag :tag => 'div', :attributes => {:id => 'user_menu_ul'}
-    assert_tag :tag => 'div', :attributes => {:id => 'user_menu_ul'}, 
+    assert_tag :tag => 'div', :attributes => {:id => 'user_menu_ul'},
                 :descendant => {:tag => 'a', :attributes => { :href => 'http://other.environment/adminuser' }},
                 :descendant => {:tag => 'a', :attributes => { :href => 'http://other.environment/myprofile/adminuser' }},
                 :descendant => {:tag => 'a', :attributes => { :href => '/admin' }}
@@ -441,4 +441,22 @@ class ApplicationControllerTest &lt; Test::Unit::TestCase
     assert_tag :html, :attributes => { :lang => 'es' }
   end
+  if ActiveRecord::Base.connection.adapter_name == 'PostgreSQL'
+
+    should 'change postgresql schema' do
+      uses_host 'schema1.com'
+      Noosfero::MultiTenancy.expects(:on?).returns(true)
+      Noosfero::MultiTenancy.expects(:mapping).returns({ 'schema1.com' => 'schema1' })
+      exception = assert_raise(ActiveRecord::StatementInvalid) { get :index }
+      assert_match /SET search_path TO schema1/, exception.message
+    end
+
+    should 'not change postgresql schema if multitenancy is off' do
+      uses_host 'schema1.com'
+      Noosfero::MultiTenancy.stubs(:on?).returns(false)
+      Noosfero::MultiTenancy.stubs(:mapping).returns({ 'schema1.com' => 'schema1' })
+      assert_nothing_raised(ActiveRecord::StatementInvalid) { get :index }
+    end
+
+  end
 end
@@ -185,6 +185,32 @@ class Test::Unit::TestCase
     end
   end
+  def uses_postgresql(schema_name = 'test_schema')
+    adapter = ActiveRecord::Base.connection.class
+    adapter.any_instance.stubs(:adapter_name).returns('PostgreSQL')
+    adapter.any_instance.stubs(:schema_search_path).returns(schema_name)
+    reload_for_ferret
+  end
+
+  def uses_sqlite
+    adapter = ActiveRecord::Base.connection.class
+    adapter.any_instance.stubs(:adapter_name).returns('SQLite')
+  end
+
+  def reload_for_ferret
+    ActsAsFerret.send(:remove_const, :DEFAULT_FIELD_OPTIONS)
+    load File.join(RAILS_ROOT, 'lib', 'acts_as_searchable.rb')
+    load File.join(RAILS_ROOT, 'vendor', 'plugins', 'acts_as_ferret', 'lib', 'acts_as_ferret.rb')
+    [Article, Profile, Product].each do |clazz|
+      inst_meth = clazz.instance_methods.reject{ |m| m =~ /_to_ferret$/ }
+      clazz.stubs(:instance_methods).returns(inst_meth)
+    end
+    #FIXME Is there a way to avoid this replication from model code?
+    Article.acts_as_searchable :additional_fields => [ :comment_data ]
+    Profile.acts_as_searchable :additional_fields => [ :extra_data_for_index ]
+    Product.acts_as_searchable :fields => [ :name, :description, :category_full_name ]
+  end
+
 end
 module NoosferoTestHelper
@@ -1503,4 +1503,27 @@ class ArticleTest &lt; Test::Unit::TestCase
     assert !child.accept_uploads?
   end
+  should 'index by schema name when database is postgresql' do
+    uses_postgresql 'schema_one'
+    art1 = Article.create!(:name => 'some thing', :profile_id => @profile.id)
+    assert_equal Article.find_by_contents('thing'), [art1]
+    uses_postgresql 'schema_two'
+    art2 = Article.create!(:name => 'another thing', :profile_id => @profile.id)
+    assert_not_includes Article.find_by_contents('thing'), art1
+    assert_includes Article.find_by_contents('thing'), art2
+    uses_postgresql 'schema_one'
+    assert_includes Article.find_by_contents('thing'), art1
+    assert_not_includes Article.find_by_contents('thing'), art2
+    uses_sqlite
+  end
+
+  should 'not index by schema name when database is not postgresql' do
+    uses_sqlite
+    art1 = Article.create!(:name => 'some thing', :profile_id => @profile.id)
+    assert_equal Article.find_by_contents('thing'), [art1]
+    art2 = Article.create!(:name => 'another thing', :profile_id => @profile.id)
+    assert_includes Article.find_by_contents('thing'), art1
+    assert_includes Article.find_by_contents('thing'), art2
+  end
+
 end
@@ -51,14 +51,6 @@ class BlockTest &lt; Test::Unit::TestCase
     assert b.cacheable?
   end
-  should 'provide chache keys' do
-     p = create_user('test_user').person
-     box = p.boxes[0]
-     b = fast_create(Block, :box_id => box.id)
-
-     assert_equal( "block-id-#{b.id}", b.cache_keys)
-  end
-
   should 'list enabled blocks' do
     block1 = fast_create(Block, :title => 'test 1')
     block2 = fast_create(Block, :title => 'test 2', :enabled => false)
@@ -95,4 +95,21 @@ class ImageTest &lt; Test::Unit::TestCase
     end
   end
+  should 'upload to a folder with same name as the schema if database is postgresql' do
+    uses_postgresql
+    file = Image.create!(:uploaded_data => fixture_file_upload('/files/rails.png', 'image/png'), :owner => profile)
+    process_delayed_job_queue
+    assert_match(/images\/test_schema\/\d{4}\/\d{4}\/rails.png/, Image.find(file.id).public_filename)
+    file.destroy
+    uses_sqlite
+  end
+
+  should 'upload to path prefix folder if database is not postgresql' do
+    uses_sqlite
+    file = Image.create!(:uploaded_data => fixture_file_upload('/files/rails.png', 'image/png'), :owner => profile)
+    process_delayed_job_queue
+    assert_match(/images\/\d{4}\/\d{4}\/rails.png/, Image.find(file.id).public_filename)
+    file.destroy
+  end
+
 end
@@ -89,7 +89,7 @@ class ProductTest &lt; Test::Unit::TestCase
   should 'be indexed by category full name' do
     p = Product.new(:name => 'a test product', :product_category => @product_category)
-    p.expects(:category_full_name).returns('interesting category')
+    p.stubs(:category_full_name).returns('interesting category')
     p.save!
     assert_includes Product.find_by_contents('interesting'), p
@@ -352,4 +352,27 @@ class ProductTest &lt; Test::Unit::TestCase
     assert_kind_of Unit, product.build_unit
   end
+  should 'index by schema name when database is postgresql' do
+    uses_postgresql 'schema_one'
+    p1 = Product.create!(:name => 'some thing', :product_category => @product_category)
+    assert_equal Product.find_by_contents('thing'), [p1]
+    uses_postgresql 'schema_two'
+    p2 = Product.create!(:name => 'another thing', :product_category => @product_category)
+    assert_not_includes Product.find_by_contents('thing'), p1
+    assert_includes Product.find_by_contents('thing'), p2
+    uses_postgresql 'schema_one'
+    assert_includes Product.find_by_contents('thing'), p1
+    assert_not_includes Product.find_by_contents('thing'), p2
+    uses_sqlite
+  end
+
+  should 'not index by schema name when database is not postgresql' do
+    uses_sqlite
+    p1 = Product.create!(:name => 'some thing', :product_category => @product_category)
+    assert_equal Product.find_by_contents('thing'), [p1]
+    p2 = Product.create!(:name => 'another thing', :product_category => @product_category)
+    assert_includes Product.find_by_contents('thing'), p1
+    assert_includes Product.find_by_contents('thing'), p2
+  end
+
 end
@@ -1668,6 +1668,29 @@ class ProfileTest &lt; Test::Unit::TestCase
     assert_equal 1, community.members_count
   end
+  should 'index by schema name when database is postgresql' do
+    uses_postgresql 'schema_one'
+    p1 = Profile.create!(:name => 'some thing', :identifier => 'some-thing')
+    assert_equal Profile.find_by_contents('thing'), [p1]
+    uses_postgresql 'schema_two'
+    p2 = Profile.create!(:name => 'another thing', :identifier => 'another-thing')
+    assert_not_includes Profile.find_by_contents('thing'), p1
+    assert_includes Profile.find_by_contents('thing'), p2
+    uses_postgresql 'schema_one'
+    assert_includes Profile.find_by_contents('thing'), p1
+    assert_not_includes Profile.find_by_contents('thing'), p2
+    uses_sqlite
+  end
+
+  should 'not index by schema name when database is not postgresql' do
+    uses_sqlite
+    p1 = Profile.create!(:name => 'some thing', :identifier => 'some-thing')
+    assert_equal Profile.find_by_contents('thing'), [p1]
+    p2 = Profile.create!(:name => 'another thing', :identifier => 'another-thing')
+    assert_includes Profile.find_by_contents('thing'), p1
+    assert_includes Profile.find_by_contents('thing'), p2
+  end
+
   private
   def assert_invalid_identifier(id)
@@ -293,4 +293,36 @@ class UploadedFileTest &lt; Test::Unit::TestCase
     assert_equal 'upload-file', UploadedFile.icon_name(f)
   end
+  should 'upload to a folder with same name as the schema if database is postgresql' do
+    uses_postgresql 'image_schema_one'
+    file1 = UploadedFile.create!(:uploaded_data => fixture_file_upload('/files/rails.png', 'image/png'), :profile => @profile)
+    process_delayed_job_queue
+    assert_match(/image_schema_one\/\d{4}\/\d{4}\/rails.png/, UploadedFile.find(file1.id).public_filename)
+    uses_postgresql 'image_schema_two'
+    file2 = UploadedFile.create!(:uploaded_data => fixture_file_upload('/files/test.txt', 'text/plain'), :profile => @profile)
+    assert_match(/image_schema_two\/\d{4}\/\d{4}\/test.txt/, UploadedFile.find(file2.id).public_filename)
+    file1.destroy
+    file2.destroy
+    uses_sqlite
+  end
+
+  should 'upload to path prefix folder if database is not postgresql' do
+    uses_sqlite
+    file = UploadedFile.create!(:uploaded_data => fixture_file_upload('/files/test.txt', 'text/plain'), :profile => @profile)
+    assert_match(/\/\d{4}\/\d{4}\/test.txt/, UploadedFile.find(file.id).public_filename)
+    assert_no_match(/test_schema\/\d{4}\/\d{4}\/test.txt/, UploadedFile.find(file.id).public_filename)
+    file.destroy
+  end
+
+  should 'upload thumbnails to a folder with same name as the schema if database is postgresql' do
+    uses_postgresql
+    file = UploadedFile.create!(:uploaded_data => fixture_file_upload('/files/rails.png', 'image/png'), :profile => @profile)
+    process_delayed_job_queue
+    UploadedFile.attachment_options[:thumbnails].each do |suffix, size|
+      assert_match(/test_schema\/\d{4}\/\d{4}\/rails_#{suffix}.png/, UploadedFile.find(file.id).public_filename(suffix))
+    end
+    file.destroy
+    uses_sqlite
+  end
+
 end
@@ -204,7 +204,7 @@ module ActsAsFerret
     # these properties are somewhat vital to the plugin and shouldn't
     # be overwritten by the user:
     index_definition[:ferret].update(
-      :key               => [:id, :class_name],
+      :key               => (ActiveRecord::Base.connection.adapter_name == 'PostgreSQL' ? [:id, :class_name, :schema_name] : [:id, :class_name]),
       :path              => index_definition[:index_dir],
       :auto_flush        => true, # slower but more secure in terms of locking problems TODO disable when running in drb mode?
       :create_if_missing => true
@@ -0,0 +1,139 @@		@@ -0,0 +1,139 @@
	1	+== Multitenancy support
	2	+
	3	+Multitenancy refers to a principle in software architecture where a
	4	+single instance of the software runs on a server, serving multiple
	5	+client organizations (tenants). Multitenancy is contrasted with a
	6	+multi-instance architecture where separate software instances (or
	7	+hardware systems) are set up for different client organizations. With
	8	+a multitenant architecture, a software application is designed to
	9	+virtually partition its data and configuration, and each client
	10	+organization works with a customized virtual application instance.
	11	+
	12	+Today this feature is available only for PostgreSQL databases.
	13	+
	14	+This document assumes that you have a fully PostgresSQL default Noosfero
	15	+installation as explained at the INSTALL file.
	16	+
	17	+== Separated data
	18	+
	19	+The items below are separated for each hosted environment:
	20	+
	21	+* Uploaded files
	22	+* Database
	23	+* Ferret index
	24	+* ActiveRecord#cache_key
	25	+* Feed updater
	26	+* Delayed Job Workers
	27	+
	28	+== Database configuration file
	29	+
	30	+The file config/database.yml must follow a structure in order to
	31	+achieve multitenancy support. In this example, we will set 3
	32	+different environments: env1, env2 and env3.
	33	+
	34	+Each "hosted" environment must have an entry like this:
	35	+
	36	+env1_production:
	37	+ adapter: postgresql
	38	+ encoding: unicode
	39	+ database: noosfero
	40	+ schema_search_path: public
	41	+ username: noosfero
	42	+ domains:
	43	+ - env1.com
	44	+ - env1.org
	45	+
	46	+env2_production:
	47	+ adapter: postgresql
	48	+ encoding: unicode
	49	+ database: noosfero
	50	+ schema_search_path: env2
	51	+ username: noosfero
	52	+ domains:
	53	+ - env2.com
	54	+ - env2.org
	55	+
	56	+env3_production:
	57	+ adapter: postgresql
	58	+ encoding: unicode
	59	+ database: noosfero
	60	+ schema_search_path: env3
	61	+ username: noosfero
	62	+ domains:
	63	+ - env3.com
	64	+ - env3.net
	65	+
	66	+The "hosted" environments define, besides the schema_search_path, a
	67	+list of domains that, when accessed, tells which database the
	68	+application should use. Also, the environment name must end with
	69	+'_hosting', where 'hosting' is the name of the hosting environment.
	70	+
	71	+You must also tell the application which is the default environment.
	72	+
	73	+production:
	74	+ env1_production
	75	+
	76	+On the example above there are only three hosted environments, but it
	77	+can be more than three. The schemas 'env2' and 'env3' must already
	78	+exist in the same database of the hosting environment. As postgres
	79	+user, you can create them typing:
	80	+
	81	+$ psql database_name -c "CREATE SCHEMA env2 AUTHORIZATION database_user"
	82	+$ psql database_name -c "CREATE SCHEMA env3 AUTHORIZATION database_user"
	83	+
	84	+Replace database_name and database_user above with your stuff.
	85	+
	86	+So, yet on this same example, when a user accesses http://env2.com or
	87	+http://env2.org, the Noosfero application running on production will
	88	+turn the database schema to 'env2'. When the access is from domains
	89	+http://env3.com or http://env3.net, the schema to be loaded will be
	90	+'env3'.
	91	+
	92	+There is an example of this file in config/database.yml.multitenancy
	93	+
	94	+== Preparing the database
	95	+
	96	+Now create the environments:
	97	+
	98	+$ RAILS_ENV=production rake multitenancy:create
	99	+
	100	+This command above will create the hosted environment files equal to
	101	+their hosting environment, here called 'production'.
	102	+
	103	+Run db:schema:load for each other environment:
	104	+
	105	+$ RAILS_ENV=env2_production rake db:schema:load
	106	+$ RAILS_ENV=env3_production rake db:schema:load
	107	+
	108	+Then run the migrations for the hosting environment, and it will
	109	+run for each of its hosted environments:
	110	+
	111	+RAILS_ENV=production rake db:migrate
	112	+
	113	+== Start Noosfero
	114	+
	115	+Run Noosfero init file as root:
	116	+
	117	+# invoke-rc.d noosfero start
	118	+
	119	+== Ferret
	120	+
	121	+It's necessary to run only one instance of ferret_server. Don't worry
	122	+about this, Noosfero initializer had already done this for you.
	123	+
	124	+Build or rebuild the Ferret index by running the following task just
	125	+for your hosting environment, do this as noosfero user:
	126	+
	127	+$ RAILS_ENV=production rake multitenancy:reindex
	128	+
	129	+== Feed updater & Delayed job
	130	+
	131	+Just for your information, a daemon of feed-updater and delayed_job
	132	+must be running for each environment. Noosfero initializer do this,
	133	+relax.
	134	+
	135	+== Uploaded files
	136	+
	137	+When running with PostgreSQL, Noosfero uploads stuff to a folder named
	138	+the same way as the running schema. Inside the upload folder root, for
	139	+example, will be public/image_uploads/env2 and public/image_uploads/env3.
	@@ -82,7 +82,7 @@ class BoxOrganizerController < ApplicationController		@@ -82,7 +82,7 @@ class BoxOrganizerController < ApplicationController
82	def save	82	def save
83	@block = boxes_holder.blocks.find(params[:id])	83	@block = boxes_holder.blocks.find(params[:id])
84	@block.update_attributes(params[:block])	84	@block.update_attributes(params[:block])
85	- expire_timeout_fragment(@block.cache_keys)	85	+ expire_timeout_fragment(@block.cache_key)
86	redirect_to :action => 'index'	86	redirect_to :action => 'index'
87	end	87	end
88		88
	@@ -93,7 +93,7 @@ class BoxOrganizerController < ApplicationController		@@ -93,7 +93,7 @@ class BoxOrganizerController < ApplicationController
93	def remove	93	def remove
94	@block = Block.find(params[:id])	94	@block = Block.find(params[:id])
95	if @block.destroy	95	if @block.destroy
96	- expire_timeout_fragment(@block.cache_keys)	96	+ expire_timeout_fragment(@block.cache_key)
97	redirect_to :action => 'index'	97	redirect_to :action => 'index'
98	else	98	else
99	session[:notice] = _('Failed to remove block')	99	session[:notice] = _('Failed to remove block')
	@@ -22,7 +22,7 @@ module SweeperHelper		@@ -22,7 +22,7 @@ module SweeperHelper
22		22
23	# friends blocks	23	# friends blocks
24	blocks = profile.blocks.select{\|b\| b.kind_of?(FriendsBlock)}	24	blocks = profile.blocks.select{\|b\| b.kind_of?(FriendsBlock)}
25	- blocks.map(&:cache_keys).each{\|ck\|expire_timeout_fragment(ck)}	25	+ blocks.map(&:cache_key).each{\|ck\|expire_timeout_fragment(ck)}
26	end	26	end
27		27
28	def expire_communities(profile)	28	def expire_communities(profile)
	@@ -34,13 +34,13 @@ module SweeperHelper		@@ -34,13 +34,13 @@ module SweeperHelper
34		34
35	# communities block	35	# communities block
36	blocks = profile.blocks.select{\|b\| b.kind_of?(CommunitiesBlock)}	36	blocks = profile.blocks.select{\|b\| b.kind_of?(CommunitiesBlock)}
37	- blocks.map(&:cache_keys).each{\|ck\|expire_timeout_fragment(ck)}	37	+ blocks.map(&:cache_key).each{\|ck\|expire_timeout_fragment(ck)}
38	end	38	end
39		39
40	def expire_enterprises(profile)	40	def expire_enterprises(profile)
41	# enterprises and favorite enterprises blocks	41	# enterprises and favorite enterprises blocks
42	blocks = profile.blocks.select {\|b\| [EnterprisesBlock, FavoriteEnterprisesBlock].any?{\|klass\| b.kind_of?(klass)} }	42	blocks = profile.blocks.select {\|b\| [EnterprisesBlock, FavoriteEnterprisesBlock].any?{\|klass\| b.kind_of?(klass)} }
43	- blocks.map(&:cache_keys).each{\|ck\|expire_timeout_fragment(ck)}	43	+ blocks.map(&:cache_key).each{\|ck\|expire_timeout_fragment(ck)}
44	end	44	end
45		45
46	def expire_profile_index(profile)	46	def expire_profile_index(profile)
	@@ -20,4 +20,6 @@ class Image < ActiveRecord::Base		@@ -20,4 +20,6 @@ class Image < ActiveRecord::Base
20		20
21	delay_attachment_fu_thumbnails	21	delay_attachment_fu_thumbnails
22		22
		23	+ postgresql_attachment_fu
		24	+
23	end	25	end
	@@ -2,4 +2,6 @@ class Thumbnail < ActiveRecord::Base		@@ -2,4 +2,6 @@ class Thumbnail < ActiveRecord::Base
2	has_attachment :storage => :file_system,	2	has_attachment :storage => :file_system,
3	:content_type => :image, :max_size => 5.megabytes	3	:content_type => :image, :max_size => 5.megabytes
4	validates_as_attachment	4	validates_as_attachment
		5	+
		6	+ postgresql_attachment_fu
5	end	7	end
	@@ -52,6 +52,8 @@ class UploadedFile < Article		@@ -52,6 +52,8 @@ class UploadedFile < Article
52		52
53	delay_attachment_fu_thumbnails	53	delay_attachment_fu_thumbnails
54		54
		55	+ postgresql_attachment_fu
		56	+
55	def self.icon_name(article = nil)	57	def self.icon_name(article = nil)
56	if article	58	if article
57	article.image? ? article.public_filename(:icon) : (article.mime_type ? article.mime_type.gsub(/[\/+.]/, '-') : 'upload-file')	59	article.image? ? article.public_filename(:icon) : (article.mime_type ? article.mime_type.gsub(/[\/+.]/, '-') : 'upload-file')
	@@ -21,7 +21,7 @@ protected		@@ -21,7 +21,7 @@ protected
21	blocks = article.profile.blocks	21	blocks = article.profile.blocks
22	blocks += article.profile.environment.blocks if article.profile.environment	22	blocks += article.profile.environment.blocks if article.profile.environment
23	blocks = blocks.select{\|b\|[RecentDocumentsBlock, BlogArchivesBlock].any?{\|c\| b.kind_of?(c)}}	23	blocks = blocks.select{\|b\|[RecentDocumentsBlock, BlogArchivesBlock].any?{\|c\| b.kind_of?(c)}}
24	- blocks.map(&:cache_keys).each{\|ck\|expire_timeout_fragment(ck)}	24	+ blocks.map(&:cache_key).each{\|ck\|expire_timeout_fragment(ck)}
25	env = article.profile.environment	25	env = article.profile.environment
26	if env && (env.portal_community == article.profile)	26	if env && (env.portal_community == article.profile)
27	expire_fragment(env.portal_news_cache_key)	27	expire_fragment(env.portal_news_cache_key)
	@@ -35,7 +35,7 @@ protected		@@ -35,7 +35,7 @@ protected
35	end	35	end
36		36
37	blocks = profile.blocks.select{\|b\| b.kind_of?(FriendsBlock)}	37	blocks = profile.blocks.select{\|b\| b.kind_of?(FriendsBlock)}
38	- blocks.map(&:cache_keys).each{\|ck\|expire_timeout_fragment(ck)}	38	+ blocks.map(&:cache_key).each{\|ck\|expire_timeout_fragment(ck)}
39	end	39	end
40		40
41	end	41	end
	@@ -23,12 +23,12 @@ protected		@@ -23,12 +23,12 @@ protected
23	expire_profile_index(profile) if profile.person?	23	expire_profile_index(profile) if profile.person?
24		24
25	profile.blocks.each do \|block\|	25	profile.blocks.each do \|block\|
26	- expire_timeout_fragment(block.cache_keys)	26	+ expire_timeout_fragment(block.cache_key)
27	end	27	end
28	end	28	end
29		29
30	def expire_statistics_block_cache(profile)	30	def expire_statistics_block_cache(profile)
31	blocks = profile.environment.blocks.select { \|b\| b.kind_of?(EnvironmentStatisticsBlock) }	31	blocks = profile.environment.blocks.select { \|b\| b.kind_of?(EnvironmentStatisticsBlock) }
32	- blocks.map(&:cache_keys).each{\|ck\|expire_timeout_fragment(ck)}	32	+ blocks.map(&:cache_key).each{\|ck\|expire_timeout_fragment(ck)}
33	end	33	end
34	end	34	end
	@@ -25,7 +25,7 @@ protected		@@ -25,7 +25,7 @@ protected
25		25
26	profile.blocks_to_expire_cache.each { \|block\|	26	profile.blocks_to_expire_cache.each { \|block\|
27	blocks = profile.blocks.select{\|b\| b.kind_of?(block)}	27	blocks = profile.blocks.select{\|b\| b.kind_of?(block)}
28	- blocks.map(&:cache_keys).each{\|ck\|expire_timeout_fragment(ck)}	28	+ blocks.map(&:cache_key).each{\|ck\|expire_timeout_fragment(ck)}
29	}	29	}
30	end	30	end
31		31