Download as
Browse Code »

Commit ecd3f31242bfecf770ad76d9cb1613fc29900188

Authored by Antonio Terceiro
1 parent 6d904169
Exists in master and in 22 other branches angular_poc, api-article-archived, article-readonly, backup-7-jan-2016, captcha_serpro_plugin, content-manager-hostspot, export-comment-paragraph, fix_event_date_issue, login-captcha, master_rails3, new_video_plugin, pg_search_rank, production, refactor_with_role, refactor_with_role_scope, remove_profile_cat_icons, resend_confirmation_email, staging, staging_rails3, tasks_keep_filter_params, theme-brasil-digital-from-staging, travis

Replacing Ruby-GetText with fast_gettext 

Besides being faster, consumming less memory, and being thread-safe,
fast_gettext's approach is cleaner than Ruby-GetText's because it does not
mess with the Rails internals. That's probably due to the fact that
fast_gettext was designed after Rails had proper I18N support, so that's
not exactly Ruby-GetText's fault. Current versions of Ruby-GetText are
claimed to be thread-safe as well, but I decided to go with fast_gettext
regardless.

I am messing with the Rails internals myself by copying some code from
Ruby-Gettext, but that code will be dropped when we upgrade to a more
recent Rails version with proper I18N. Code was copied from Ruby-GetText
to implement:

    * per-language cache
    * validation error messages translation

During initialization, the needed .mo files installed system-wide are
symlinked locally.  By doing this we can take "similar" locales locally
since fast_gettext does not seem to support loading of files from similar
locales (e.g.  loading pt_BR/LC_MESSAGES/domain.mo when
pt/LC_MESSAGES/domain.mo is not available).

This hopefully will fix the long-standing bug with messed up translations
due to high concurrency and non-thread-safety of the version of
Ruby-GetText in Debian Lenny.

(ActionItem1315)
Showing 24 changed files with 226 additions and 80 deletions   Show diff stats
app/controllers/application.rb
  Diff comments   View file @ecd3f31
... ... @@ -70,20 +70,15 @@ class ApplicationController < ActionController::Base
70 70 end
71 71 end
72 72  
73   - include GetText
74   - before_init_gettext :maybe_save_locale, :default_locale
75   - def maybe_save_locale
  73 + before_filter :set_locale
  74 + def set_locale
  75 + FastGettext.available_locales = Noosfero.available_locales
  76 + FastGettext.default_locale = Noosfero.default_locale
  77 + FastGettext.set_locale(params[:lang] || session[:lang] || Noosfero.default_locale || request.env['HTTP_ACCEPT_LANGUAGE'] || 'en')
76 78 if params[:lang]
77   - cookies[:lang] = params[:lang]
  79 + session[:lang] = params[:lang]
78 80 end
79 81 end
80   - def default_locale
81   - if Noosfero.default_locale && cookies[:lang].blank?
82   - cookies[:lang] = params[:lang] = Noosfero.default_locale
83   - end
84   - end
85   - protected :maybe_save_locale, :default_locale
86   - init_gettext 'noosfero'
87 82  
88 83 include NeedsProfile
89 84  
... ...
app/helpers/account_helper.rb
  Diff comments   View file @ecd3f31
1 1 module AccountHelper
2 2  
3   - include GetText
4 3  
5 4 def button_to_step(type, step, current_step, html_options = {})
6 5 if current_step == step
... ...
app/helpers/application_helper.rb
  Diff comments   View file @ecd3f31
... ... @@ -26,6 +26,10 @@ module ApplicationHelper
26 26  
27 27 include AccountHelper
28 28  
  29 + def locale
  30 + FastGettext.locale
  31 + end
  32 +
29 33 def load_web2_conf
30 34 if File.exists?( RAILS_ROOT + '/config/web2.0.yml')
31 35 YAML.load_file( RAILS_ROOT + '/config/web2.0.yml' )
... ... @@ -664,7 +668,6 @@ module ApplicationHelper
664 668  
665 669 # Should be on the forms_helper file but when its there the translation of labels doesn't work
666 670 class NoosferoFormBuilder < ActionView::Helpers::FormBuilder
667   - include GetText
668 671 extend ActionView::Helpers::TagHelper
669 672  
670 673 def self.output_field(text, field_html, field_id = nil)
... ... @@ -687,13 +690,7 @@ module ApplicationHelper
687 690 (field_helpers - %w(hidden_field)).each do |selector|
688 691 src = <<-END_SRC
689 692 def #{selector}(field, *args, &proc)
690   - column = object.class.columns_hash[field.to_s]
691   - text =
692   - ( column ?
693   - column.human_name :
694   - field.to_s.humanize
695   - )
696   -
  693 + text = object.class.human_attribute_name(field.to_s)
697 694 NoosferoFormBuilder::output_field(text, super)
698 695 end
699 696 END_SRC
... ...
app/helpers/categories_helper.rb
  Diff comments   View file @ecd3f31
1 1 module CategoriesHelper
2 2  
3   - include GetText
4 3  
5 4 COLORS = [
6 5 [ N_('Do not display at the menu'), nil ],
... ...
app/helpers/content_viewer_helper.rb
  Diff comments   View file @ecd3f31
1 1 module ContentViewerHelper
2 2  
3   - include GetText
4 3 include BlogHelper
5 4  
6 5 def number_of_comments(article)
... ...
app/helpers/countries_helper.rb
  Diff comments   View file @ecd3f31
... ... @@ -2,9 +2,6 @@ class CountriesHelper
2 2  
3 3 include Singleton
4 4  
5   - include GetText
6   - bindtextdomain 'iso_3166'
7   -
8 5 # a dump of iso_3166.xml from Debian source package iso-codes
9 6 COUNTRIES = [
10 7 ["Afghanistan", "AF"],
... ...
app/helpers/dates_helper.rb
  Diff comments   View file @ecd3f31
1   -module DatesHelper
  1 +require 'noosfero/i18n'
2 2  
3   - include GetText
  3 +module DatesHelper
4 4  
5 5 # FIXME Date#strftime should translate this for us !!!!
6 6 MONTHS = [
... ...
app/helpers/language_helper.rb
  Diff comments   View file @ecd3f31
1 1 module LanguageHelper
2 2 def language
3   - if Noosfero.available_locales.include?(locale.to_s) ||
4   - Noosfero.available_locales.include?(locale.language)
5   - locale.language
6   - else
7   - Noosfero.default_locale || 'en'
8   - end
  3 + locale
9 4 end
10 5  
11 6 def tinymce_language
... ...
app/helpers/profile_editor_helper.rb
  Diff comments   View file @ecd3f31
1 1 module ProfileEditorHelper
2 2  
3   - include GetText
4 3  
5 4 AREAS_OF_STUDY = [
6 5 N_('Agrometeorology'),
... ...
app/models/task.rb
  Diff comments   View file @ecd3f31
... ... @@ -12,7 +12,6 @@
12 12 class Task < ActiveRecord::Base
13 13  
14 14 module Status
15   - include GetText
16 15 # the status of tasks just created
17 16 ACTIVE = 1
18 17  
... ...
app/views/layouts/application-ng.rhtml
  Diff comments   View file @ecd3f31
... ... @@ -37,7 +37,7 @@
37 37 }
38 38 </script>
39 39  
40   - <a href="#content" id="link-go-content"><span>Ir para o conteúdo</span></a>
  40 + <a href="#content" id="link-go-content"><span><%= _("Go to the content") %></span></a>
41 41  
42 42 <div id="wrap-1">
43 43 <div id='theme-header'>
... ...
config/environment.rb
  Diff comments   View file @ecd3f31
... ... @@ -10,17 +10,6 @@ RAILS_GEM_VERSION = &#39;2.1.0&#39; unless defined? RAILS_GEM_VERSION
10 10 # Bootstrap the Rails environment, frameworks, and default configuration
11 11 require File.join(File.dirname(__FILE__), 'boot')
12 12  
13   -# locally-developed modules
14   -require 'locale'
15   -require 'gettext/rails'
16   -require 'acts_as_filesystem'
17   -require 'acts_as_having_settings'
18   -require 'acts_as_searchable'
19   -require 'acts_as_having_boxes'
20   -require 'acts_as_having_image'
21   -require 'will_paginate'
22   -require 'route_if'
23   -
24 13 # extra directories for controllers organization
25 14 extra_controller_dirs = %w[
26 15 app/controllers/my_profile
... ...
config/initializers/dependencies.rb 0 → 100644
  Diff comments   View file @ecd3f31
... ... @@ -0,0 +1,10 @@
  1 +# locally-developed modules
  2 +require 'acts_as_filesystem'
  3 +require 'acts_as_having_settings'
  4 +require 'acts_as_searchable'
  5 +require 'acts_as_having_boxes'
  6 +require 'acts_as_having_image'
  7 +require 'route_if'
  8 +
  9 +# third-party libraries
  10 +require 'will_paginate'
... ...
config/initializers/fast_gettext.rb 0 → 100644
  Diff comments   View file @ecd3f31
... ... @@ -0,0 +1 @@
  1 +require 'noosfero/i18n'
... ...
features/step_definitions/internationalization_steps.rb
  Diff comments   View file @ecd3f31
1   -def language_to_header(name)
  1 +def language_to_code(name)
2 2 {
3 3 'Brazilian Portuguese' => 'pt-br',
4 4 'European Portuguese' => 'pt-pt',
... ... @@ -17,29 +17,25 @@ def native_name(name)
17 17 }[name] || name
18 18 end
19 19  
20   -def language_to_code(name)
21   - language_to_header(name)
22   -end
23   -
24 20 Given /^Noosfero is configured to use (.+) as default$/ do |lang|
25 21 Noosfero.default_locale = language_to_code(lang)
26 22 end
27 23  
28   -After('@default_locale_config') do
  24 +After do
  25 + # reset everything back to normal
29 26 Noosfero.default_locale = nil
  27 + FastGettext.locale = 'en'
30 28 end
31 29  
32 30 Given /^a user accessed in (.*) before$/ do |lang|
33 31 session = Webrat::Session.new(Webrat.adapter_class.new(self))
34 32 session.extend(Webrat::Matchers)
35 33 session.visit('/')
36   - session.should have_selector('html[lang=en]')
  34 + session.should have_selector("html[lang=#{language_to_code(lang)}]")
37 35 end
38 36  
39 37 Given /^my browser prefers (.*)$/ do |lang|
40   - @n ||= 0
41   - header 'Accept-Language', language_to_header(lang)
42   -
  38 + header 'Accept-Language', language_to_code(lang)
43 39 end
44 40  
45 41 Then /^the site should be in (.*)$/ do |lang|
... ...
lib/noosfero.rb
  Diff comments   View file @ecd3f31
... ... @@ -12,12 +12,15 @@ module Noosfero
12 12 attr_accessor :locales
13 13 attr_accessor :default_locale
14 14 def available_locales
15   - @available_locales ||= locales.keys
16   - end
17   - def each_locale
18   - locales.keys.sort.each do |key|
19   - yield(key, locales[key])
20   - end
  15 + @available_locales ||=
  16 + begin
  17 + locales_list = locales.keys
  18 + # move English to the beginning
  19 + if locales_list.include?('en')
  20 + locales_list = ['en'] + (locales_list - ['en']).sort
  21 + end
  22 + locales_list
  23 + end
21 24 end
22 25 end
23 26  
... ...
lib/noosfero/i18n.rb 0 → 100644
  Diff comments   View file @ecd3f31
... ... @@ -0,0 +1,183 @@
  1 +require 'fast_gettext'
  2 +
  3 +class Object
  4 + include FastGettext::Translation
  5 + alias :gettext :_
  6 + alias :ngettext :n_
  7 +end
  8 +
  9 +class ActiveRecord::Errors
  10 + default_error_messages.update(
  11 + :inclusion => N_("%{fn} is not included in the list"),
  12 + :exclusion => N_("%{fn} is reserved"),
  13 + :invalid => N_("%{fn} is invalid"),
  14 + :confirmation => N_("%{fn} doesn't match confirmation"),
  15 + :accepted => N_("%{fn} must be accepted"),
  16 + :empty => N_("%{fn} can't be empty"),
  17 + :blank => N_("%{fn} can't be blank"),
  18 + :too_long => N_("%{fn} is too long (maximum is %d characters)"),
  19 + :too_short => N_("%{fn} is too short (minimum is %d characters)"),
  20 + :wrong_length => N_("%{fn} is the wrong length (should be %d characters)"),
  21 + :taken => N_("%{fn} has already been taken"),
  22 + :not_a_number => N_("%{fn} is not a number")
  23 + )
  24 +
  25 + def localize_error_messages
  26 + errors = {}
  27 + each do |attr,msg|
  28 + next if msg.nil?
  29 + errors[attr] ||= []
  30 + errors[attr] << _(msg).sub('%{fn}', @base.class.human_attribute_name(attr))
  31 + end
  32 + errors
  33 + end
  34 + def on_with_gettext(attribute)
  35 + errors = localize_error_messages[attribute.to_s]
  36 + return nil if errors.nil?
  37 + errors.size == 1 ? errors.first : errors
  38 + end
  39 + alias_method_chain :on, :gettext
  40 +
  41 + def full_messages_with_gettext
  42 + full_messages = []
  43 + errors = localize_error_messages
  44 + errors.each_key do |attr|
  45 + errors[attr].each do |msg|
  46 + next if msg.nil?
  47 + full_messages << msg
  48 + end
  49 + end
  50 + full_messages
  51 + end
  52 + alias_method_chain :full_messages, :gettext
  53 +end
  54 +
  55 +
  56 +module ActionView::Helpers::ActiveRecordHelper
  57 + module L10n
  58 + @error_message_title = Nn_("%{num} error prohibited this %{record} from being saved", "%{num} errors prohibited this %{record} from being saved").flatten
  59 + @error_message_explanation = Nn_("There was a problem with the following field:", "There were problems with the following fields:").flatten
  60 + module_function
  61 + def error_messages_for(instance, objects, object_names, count, options)
  62 + record = _(options[:object_name] || object_names[0].to_s)
  63 +
  64 + html = {}
  65 + [:id, :class].each do |key|
  66 + if options.include?(key)
  67 + value = options[key]
  68 + html[key] = value unless value.blank?
  69 + else
  70 + html[key] = 'errorExplanation'
  71 + end
  72 + end
  73 +
  74 + if options[:message_title]
  75 + header_message = instance.error_message(options[:message_title], count) % {:num => count, :record => record}
  76 + else
  77 + header_message = ((count == 1) ? _(@error_message_title[0]) : _(@error_message_title[1])) % {:num => count, :record => record}
  78 + end
  79 + if options[:message_explanation]
  80 + message_explanation = instance.error_message(options[:message_explanation], count) % {:num => count}
  81 + else
  82 + message_explanation = (count == 1 ? _(@error_message_explanation[0]) : _(@error_message_explanation[1])) % {:num => count}
  83 + end
  84 +
  85 + error_messages = objects.map {|object| object.errors.full_messages.map {|msg| instance.content_tag(:li, msg) } }
  86 +
  87 + instance.content_tag(
  88 + :div,
  89 + instance.content_tag(options[:header_tag] || :h2, header_message) <<
  90 + instance.content_tag(:p, message_explanation) <<
  91 + instance.content_tag(:ul, error_messages),
  92 + html
  93 + )
  94 + end
  95 + end
  96 +
  97 + alias error_messages_for_without_localize error_messages_for #:nodoc:
  98 +
  99 + # error_messages_for overrides original method with localization.
  100 + # And also it extends to be able to replace the title/explanation of the header of the error dialog. (Since 1.90)
  101 + # If you want to override these messages in the whole application,
  102 + # use ActionView::Helpers::ActiveRecordHelper::L10n.set_error_message_(title|explanation) instead.
  103 + # * :message_title - the title of message. Use Nn_() to path the strings for singular/plural.
  104 + # e.g. Nn_("%{num} error prohibited this %{record} from being saved",
  105 + # "%{num} errors prohibited this %{record} from being saved")
  106 + # * :message_explanation - the explanation of message
  107 + # e.g. Nn_("There was a problem with the following field:",
  108 + # "There were %{num} problems with the following fields:")
  109 + def error_messages_for(*params)
  110 + options = params.last.is_a?(Hash) ? params.pop.symbolize_keys : {}
  111 + objects = params.collect {|object_name| instance_variable_get("@#{object_name}") }.compact
  112 + object_names = params.dup
  113 + count = objects.inject(0) {|sum, object| sum + object.errors.count }
  114 + if count.zero?
  115 + ''
  116 + else
  117 + L10n.error_messages_for(self, objects, object_names, count, options)
  118 + end
  119 + end
  120 +
  121 +end
  122 +
  123 +module ActionController::Caching::Fragments
  124 + def fragment_cache_key_with_fast_gettext(name)
  125 + ret = fragment_cache_key_without_fast_gettext(name)
  126 + if ret.is_a? String
  127 + ret.gsub(/:/, ".") << "_#{FastGettext.locale}"
  128 + else
  129 + ret
  130 + end
  131 + end
  132 + alias_method_chain :fragment_cache_key, :fast_gettext
  133 +
  134 + def expire_fragment_with_fast_gettext(name, options = nil)
  135 + return unless perform_caching
  136 +
  137 + key = fragment_cache_key_without_fast_gettext(name)
  138 + if key.is_a?(Regexp)
  139 + self.class.benchmark "Expired fragments matching: #{key.source}" do
  140 + fragment_cache_store.delete_matched(key, options)
  141 + end
  142 + else
  143 + key = key.gsub(/:/, ".")
  144 + self.class.benchmark "Expired fragment: #{key}, lang = #{FastGettext.available_locales.inspect}" do
  145 + if FastGettext.available_locales
  146 + FastGettext.available_locales.each do |lang|
  147 + fragment_cache_store.delete("#{key}_#{lang}", options)
  148 + end
  149 + end
  150 + end
  151 + end
  152 + end
  153 + alias_method_chain :expire_fragment, :fast_gettext
  154 +end
  155 +
  156 +FileUtils.mkdir_p(Rails.root + '/locale')
  157 +Dir.glob(Rails.root + '/locale/*').each do |dir|
  158 + lang = File.basename(dir)
  159 + FileUtils.mkdir_p("#{Rails.root}/locale/#{lang}/LC_MESSAGES")
  160 + ['iso_3166', 'rails'].each do |domain|
  161 + target = "#{Rails.root}/locale/#{lang}/LC_MESSAGES/#{domain}.mo"
  162 + if !File.exists?(target)
  163 + orig = "/usr/share/locale/#{lang}/LC_MESSAGES/#{domain}.mo"
  164 + if File.exists?(orig)
  165 + File.symlink(orig, target)
  166 + else
  167 + alternatives = Dir.glob("/usr/share/locale/#{lang}_*/LC_MESSAGES/#{domain}.mo")
  168 + unless alternatives.empty?
  169 + File.symlink(alternatives.first, target)
  170 + end
  171 + end
  172 + end
  173 + end
  174 +end
  175 +
  176 +repos = [
  177 + FastGettext::TranslationRepository.build('noosfero', :type => 'mo', :path => Rails.root + '/locale'),
  178 + FastGettext::TranslationRepository.build('iso_3166', :type => 'mo', :path => Rails.root + '/locale'),
  179 + FastGettext::TranslationRepository.build('rails', :type => 'mo', :path => Rails.root + '/locale'),
  180 +]
  181 +
  182 +FastGettext.add_text_domain 'noosferofull', :type => :chain, :chain => repos
  183 +FastGettext.default_text_domain = 'noosferofull'
... ...
lib/tasks/populate.rake
  Diff comments   View file @ecd3f31
1 1 require File.dirname(__FILE__) + '/../../config/environment'
2 2 require 'noosfero'
3   -require 'gettext/rails'
4   -include GetText
5 3  
6 4 DEFAULT_ENVIRONMENT_TEXT = <<EOF
7 5 <h1>Environment homepage</h1>
... ...
lib/unifreire_terminology.rb
  Diff comments   View file @ecd3f31
1 1 require 'noosfero/terminology'
2 2  
3 3 class UnifreireTerminology < Noosfero::Terminology::Custom
4   - include GetText
5 4  
6 5 def initialize
7 6 # NOTE: the hash values must be marked for translation!!
... ...
lib/zen3_terminology.rb
  Diff comments   View file @ecd3f31
1 1 require 'noosfero/terminology'
2 2  
3 3 class Zen3Terminology < Noosfero::Terminology::Custom
4   - include GetText
5 4  
6 5 def initialize
7 6 # NOTE: the hash values must be marked for translation!!
... ...
test/unit/communities_block_test.rb
  Diff comments   View file @ecd3f31
... ... @@ -2,8 +2,6 @@ require File.dirname(__FILE__) + &#39;/../test_helper&#39;
2 2  
3 3 class CommunitiesBlockTest < Test::Unit::TestCase
4 4  
5   - include GetText
6   -
7 5 should 'inherit from ProfileListBlock' do
8 6 assert_kind_of ProfileListBlock, CommunitiesBlock.new
9 7 end
... ...
test/unit/enterprises_block_test.rb
  Diff comments   View file @ecd3f31
... ... @@ -2,8 +2,6 @@ require File.dirname(__FILE__) + &#39;/../test_helper&#39;
2 2  
3 3 class EnterprisesBlockTest < Test::Unit::TestCase
4 4  
5   - include GetText
6   -
7 5 should 'inherit from ProfileListBlock' do
8 6 assert_kind_of ProfileListBlock, EnterprisesBlock.new
9 7 end
... ...
test/unit/friends_block_test.rb
  Diff comments   View file @ecd3f31
... ... @@ -2,8 +2,6 @@ require File.dirname(__FILE__) + &#39;/../test_helper&#39;
2 2  
3 3 class FriendsBlockTest < ActiveSupport::TestCase
4 4  
5   - include GetText
6   -
7 5 should 'describe itself' do
8 6 assert_not_equal ProfileListBlock.description, FriendsBlock.description
9 7 end
... ...
test/unit/language_helper_test.rb
  Diff comments   View file @ecd3f31
... ... @@ -4,14 +4,9 @@ class LanguageHelperTest &lt; Test::Unit::TestCase
4 4  
5 5 include LanguageHelper
6 6  
7   -
8 7 should 'return current language' do
9   - locale = mock
10   - locale.stubs(:to_s).returns('pt_BR')
11   - locale.stubs(:language).returns('pt')
12   - stubs(:locale).returns(locale)
13   -
14   - assert_equal 'pt', self.language
  8 + expects(:locale).returns('pt')
  9 + assert_equal 'pt', language
15 10 end
16 11  
17 12 should 'remove country code for TinyMCE' do
... ...