Commit f861ffd84bc7a2bf848a1523d45fb9b66f33229a

Authored by Macartur Sousa
1 parent 020591c2

Adding elasticsearch plugin.

Create elasticsearch structure
Index models for elasticsearch
Adding category search filter
Make query generic
Add README.md for elasticsearch plugin
Start tests with rspec for elasticsearch
Applyng filtering by type on articles
Adding visible filter

Signed-off-by: Lucas Moura <lucas.moura128@gmail.com>
Signed-off-by: Luciano Prestes Cavalcanti <lucianopcbr@gmail.com>
Signed-off-by: Macartur Sousa <macartur.sc@gmail.com>
Signed-off-by: Daniel Henrique <danielhmarinho@gmail.com>
plugins/elasticsearch/Gemfile 0 → 100644
... ... @@ -0,0 +1,2 @@
  1 +gem 'elasticsearch-model'
  2 +gem 'elasticsearch-rails'
... ...
plugins/elasticsearch/README.md 0 → 100644
... ... @@ -0,0 +1,41 @@
  1 +Elasticsearch Plugin
  2 +====================
  3 +
  4 +Elasticsearch is as plugin to run searchs in noosfero through elasticsearch.
  5 +
  6 +The Version used is 1.7.5 due compatibility problems with gems and new versions.
  7 +
  8 +Download: https://www.elastic.co/downloads/past-releases/elasticsearch-1-7-5
  9 +
  10 +INSTALL
  11 +=======
  12 +
  13 +Install dependencies
  14 +
  15 +Install elasticsearch package and start service.
  16 +By default, the service runs on port 9200
  17 +
  18 +Install gems listed in plugin Gemfile. If this step fail, just copy the gems to core Gemfile
  19 +and run the command 'bundle install'
  20 +
  21 +Enable plugin
  22 +-------------
  23 +
  24 +Execute the command to enable Elasticsearch Plugin at your noosfero:
  25 +
  26 +./script/noosfero-plugins enable elasticsearch
  27 +
  28 +Active plugin
  29 +-------------
  30 +
  31 +As a Noosfero administrator user, go to administrator panel:
  32 +
  33 +- Click on "Enable/disable plugins" option
  34 +- Click on "Elasticsearch" check-box
  35 +
  36 +DEVELOPMENT
  37 +===========
  38 +
  39 +To run tests for Elasticsearch:
  40 +
  41 +Use command 'rspec'
... ...
plugins/elasticsearch/controllers/elasticsearch_plugin_controller.rb 0 → 100644
... ... @@ -0,0 +1,64 @@
  1 +class ElasticsearchPluginController < ApplicationController
  2 + no_design_blocks
  3 +
  4 + def search
  5 + @results = []
  6 + @query = params[:q]
  7 + @checkbox = {}
  8 + terms = get_terms(params)
  9 + puts "=" * 80, terms, "=" * 80
  10 + if params[:model].present?
  11 + params[:model].keys.each do |model|
  12 + @checkbox[model.to_sym] = true
  13 + klass = model.classify.constantize
  14 + query = get_query params[:q], klass
  15 + @results += klass.__elasticsearch__.search(query).records.to_a
  16 + end
  17 + end
  18 +
  19 + if params[:filter].present?
  20 + params[:filter].keys.each do |model|
  21 + params[:filter][model].keys.each do |filter|
  22 + @checkbox[filter.to_sym] = true
  23 + end
  24 + end
  25 + end
  26 + end
  27 +
  28 + private
  29 +
  30 + def get_query text, klass
  31 + query = {}
  32 + unless text.blank?
  33 +
  34 + fields = klass::SEARCHABLE_FIELDS.map {|k, v| "#{k}^#{v[:weight]}"}
  35 +
  36 + query = {
  37 + query: {
  38 + multi_match: {
  39 + query: text,
  40 + fields: fields,
  41 + operator: "and"
  42 + }
  43 + },
  44 + filter: {
  45 + term: {visible: "true"}
  46 + }
  47 + }
  48 + end
  49 + query
  50 + end
  51 +
  52 + def get_terms params
  53 + terms = {}
  54 + return terms unless params[:filter].present?
  55 + params[:filter].keys.each do |model|
  56 + terms[model] = {}
  57 + params[:filter][model].keys.each do |filter|
  58 + @checkbox[filter.to_sym] = true
  59 + terms[model][params[:filter][model.to_sym][filter]] = filter
  60 + end
  61 + end
  62 + terms
  63 + end
  64 +end
... ...
plugins/elasticsearch/lib/elasticsearch_plugin.rb 0 → 100644
... ... @@ -0,0 +1,48 @@
  1 +class ElasticsearchPlugin < Noosfero::Plugin
  2 +
  3 + def self.plugin_name
  4 + # FIXME
  5 + "ElasticsearchPlugin"
  6 + end
  7 +
  8 + def self.plugin_description
  9 + # FIXME
  10 + _("A plugin that does this and that.")
  11 + end
  12 +
  13 + Noosfero::Application.class_eval do
  14 + config.after_initialize do
  15 +
  16 + Rails.application.eager_load! #TODO: REMOVE THIS LINE
  17 +
  18 + models = ActiveRecord::Base.descendants.select do |model|
  19 + model.const_defined?("SEARCHABLE_FIELDS")
  20 + end
  21 +
  22 + models.each do |model|
  23 + model.class_eval do
  24 + include Elasticsearch::Model
  25 + include Elasticsearch::Model::Callbacks
  26 +
  27 + settings index: { number_of_shards: 1 } do
  28 + mappings dynamic: 'false' do
  29 + model::SEARCHABLE_FIELDS.each do |field, value|
  30 + indexes field
  31 + end
  32 + end
  33 + model.__elasticsearch__.client.indices.delete \
  34 + index: model.index_name rescue nil
  35 + model.__elasticsearch__.client.indices.create \
  36 + index: model.index_name,
  37 + body: {
  38 + settings: model.settings.to_hash,
  39 + mappings: model.mappings.to_hash
  40 + }
  41 +
  42 + model.import
  43 + end
  44 + end
  45 + end
  46 + end
  47 + end
  48 +end
... ...
plugins/elasticsearch/po/elasticsearch.pot 0 → 100644
... ... @@ -0,0 +1,22 @@
  1 +# SOME DESCRIPTIVE TITLE.
  2 +# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
  3 +# This file is distributed under the same license as the PACKAGE package.
  4 +# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
  5 +#
  6 +#, fuzzy
  7 +msgid ""
  8 +msgstr ""
  9 +"Project-Id-Version: 1.3~rc2-1-ga15645d\n"
  10 +"POT-Creation-Date: 2015-10-30 16:35-0300\n"
  11 +"PO-Revision-Date: 2015-08-06 17:21-0300\n"
  12 +"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
  13 +"Language-Team: LANGUAGE <LL@li.org>\n"
  14 +"Language: \n"
  15 +"MIME-Version: 1.0\n"
  16 +"Content-Type: text/plain; charset=UTF-8\n"
  17 +"Content-Transfer-Encoding: 8bit\n"
  18 +"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"
  19 +
  20 +#: plugins/template/lib/template_plugin.rb:10
  21 +msgid "A plugin that does this and that."
  22 +msgstr ""
... ...
plugins/elasticsearch/test/unit/index_models_test.rb 0 → 100644
... ... @@ -0,0 +1,21 @@
  1 +require 'test_helper'
  2 +
  3 +class IndexModelsTest < ActiveSupport::TestCase
  4 +
  5 + should "check index models on elasticsearch" do
  6 + fields = []
  7 + mappings = []
  8 +
  9 + ActiveRecord::Base.descendants.each do |model|
  10 + if model.const_defined?("SEARCHABLE_FIELDS")
  11 + mappings << model.mappings.instance_values['mapping'].keys.sort
  12 + fields << model::SEARCHABLE_FIELDS.keys.sort
  13 + end
  14 + end
  15 +
  16 + mappings.count.times do |i|
  17 + assert_equal mappings[i], fields[i]
  18 + end
  19 + end
  20 +
  21 +end
... ...
plugins/elasticsearch/views/elasticsearch_plugin/_article_display.html.erb 0 → 100644
... ... @@ -0,0 +1,2 @@
  1 +
  2 +Article: <%= article.name %>
... ...
plugins/elasticsearch/views/elasticsearch_plugin/_community_display.html.erb 0 → 100644
... ... @@ -0,0 +1 @@
  1 +Community: <%= community.name %>
... ...
plugins/elasticsearch/views/elasticsearch_plugin/_person_display.html.erb 0 → 100644
... ... @@ -0,0 +1,2 @@
  1 +
  2 +Person: <%= person.name %>
... ...
plugins/elasticsearch/views/elasticsearch_plugin/search.html.erb 0 → 100644
... ... @@ -0,0 +1,36 @@
  1 +<h1> Search </h1>
  2 +
  3 +<%= form_tag controller: "elasticsearch_plugin", action: "search" do %>
  4 + <%= label_tag(:q, _("Search")) %>
  5 + <%= text_field_tag(:q, @query) %>
  6 +
  7 + <%= submit_tag _("Send") %>
  8 +
  9 + <%= check_box_tag 'model[communities]', 1, @checkbox[:communities] %>
  10 + <%= label_tag('communities', _("communities")) %>
  11 +
  12 + <%= check_box_tag 'model[people]', 1, @checkbox[:people] %>
  13 + <%= label_tag('people', _("people")) %>
  14 +
  15 + <%= check_box_tag 'model[articles]', 1, @checkbox[:articles] %>
  16 + <%= label_tag('articles', _("articles")) %>
  17 +
  18 + <%= check_box_tag 'filter[articles][gallery]', :type, @checkbox[:gallery] %>
  19 + <%= label_tag('gallery', _("gallery")) %>
  20 +<% end %>
  21 +
  22 +<% for result in @results %>
  23 + <% if result.is_a? Article %>
  24 + <%= render partial: "article_display", :locals => {:article => result} %>
  25 + <br>
  26 + <% end %>
  27 +
  28 + <% if result.is_a? Person %>
  29 + <%= render partial: "person_display", :locals => {:person => result} %>
  30 + <br>
  31 + <% end %>
  32 + <% if result.is_a? Community %>
  33 + <%= render partial: "community_display", :locals => {:community => result} %>
  34 + <br>
  35 + <% end %>
  36 +<% end %>
... ...
spec/models/community_spec.rb 0 → 100644
... ... @@ -0,0 +1,38 @@
  1 +require 'rails_helper'
  2 +require 'rake'
  3 +require 'elasticsearch/extensions/test/cluster/tasks'
  4 +
  5 +RSpec.configure do |config|
  6 + config.before :each, elasticsearch: true do
  7 + puts '='*10, 'before', '='*10
  8 + Elasticsearch::Extensions::Test::Cluster.start() unless Elasticsearch::Extensions::Test::Cluster.running?
  9 + end
  10 +
  11 + config.after :suite do
  12 + puts '='*10, 'after', '='*10
  13 + Elasticsearch::Extensions::Test::Cluster.stop() if Elasticsearch::Extensions::Test::Cluster.running?
  14 + end
  15 +end
  16 +
  17 +RSpec.describe Community, type: :model, elasticsearch: true do
  18 + before do
  19 + Environment.create!(:name => 'Noosfero', :contact_email => 'noosfero@localhost.localdomain', :is_default => true)
  20 +
  21 + @environment = Environment.default
  22 + @environment.enabled_plugins = ['ElasticsearchPlugin']
  23 + @environment.save!
  24 +
  25 + @community = Community.new(name: "Debian")
  26 + @community.save!
  27 +
  28 + sleep 2
  29 + end
  30 +
  31 + it "assert true" do
  32 + communities = Community.__elasticsearch__.search({}).records.to_a
  33 +
  34 + p communities
  35 +
  36 + expect(true).to be true
  37 + end
  38 +end
... ...
spec/rails_helper.rb 0 → 100644
... ... @@ -0,0 +1,52 @@
  1 +# This file is copied to spec/ when you run 'rails generate rspec:install'
  2 +ENV['RAILS_ENV'] ||= 'test'
  3 +require File.expand_path('../../config/environment', __FILE__)
  4 +# Prevent database truncation if the environment is production
  5 +abort("The Rails environment is running in production mode!") if Rails.env.production?
  6 +require 'spec_helper'
  7 +require 'rspec/rails'
  8 +# Add additional requires below this line. Rails is not loaded until this point!
  9 +
  10 +# Requires supporting ruby files with custom matchers and macros, etc, in
  11 +# spec/support/ and its subdirectories. Files matching `spec/**/*_spec.rb` are
  12 +# run as spec files by default. This means that files in spec/support that end
  13 +# in _spec.rb will both be required and run as specs, causing the specs to be
  14 +# run twice. It is recommended that you do not name files matching this glob to
  15 +# end with _spec.rb. You can configure this pattern with the --pattern
  16 +# option on the command line or in ~/.rspec, .rspec or `.rspec-local`.
  17 +#
  18 +# The following line is provided for convenience purposes. It has the downside
  19 +# of increasing the boot-up time by auto-requiring all files in the support
  20 +# directory. Alternatively, in the individual `*_spec.rb` files, manually
  21 +# require only the support files necessary.
  22 +#
  23 +# Dir[Rails.root.join('spec/support/**/*.rb')].each { |f| require f }
  24 +
  25 +# Checks for pending migrations before tests are run.
  26 +# If you are not using ActiveRecord, you can remove this line.
  27 +ActiveRecord::Migration.maintain_test_schema!
  28 +
  29 +RSpec.configure do |config|
  30 + # Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
  31 + config.fixture_path = "#{::Rails.root}/spec/fixtures"
  32 +
  33 + # If you're not using ActiveRecord, or you'd prefer not to run each of your
  34 + # examples within a transaction, remove the following line or assign false
  35 + # instead of true.
  36 + config.use_transactional_fixtures = true
  37 +
  38 + # RSpec Rails can automatically mix in different behaviours to your tests
  39 + # based on their file location, for example enabling you to call `get` and
  40 + # `post` in specs under `spec/controllers`.
  41 + #
  42 + # You can disable this behaviour by removing the line below, and instead
  43 + # explicitly tag your specs with their type, e.g.:
  44 + #
  45 + # RSpec.describe UsersController, :type => :controller do
  46 + # # ...
  47 + # end
  48 + #
  49 + # The different available types are documented in the features, such as in
  50 + # https://relishapp.com/rspec/rspec-rails/docs
  51 + config.infer_spec_type_from_file_location!
  52 +end
... ...
spec/spec_helper.rb 0 → 100644
... ... @@ -0,0 +1,92 @@
  1 +# This file was generated by the `rails generate rspec:install` command. Conventionally, all
  2 +# specs live under a `spec` directory, which RSpec adds to the `$LOAD_PATH`.
  3 +# The generated `.rspec` file contains `--require spec_helper` which will cause
  4 +# this file to always be loaded, without a need to explicitly require it in any
  5 +# files.
  6 +#
  7 +# Given that it is always loaded, you are encouraged to keep this file as
  8 +# light-weight as possible. Requiring heavyweight dependencies from this file
  9 +# will add to the boot time of your test suite on EVERY test run, even for an
  10 +# individual file that may not need all of that loaded. Instead, consider making
  11 +# a separate helper file that requires the additional dependencies and performs
  12 +# the additional setup, and require it from the spec files that actually need
  13 +# it.
  14 +#
  15 +# The `.rspec` file also contains a few flags that are not defaults but that
  16 +# users commonly want.
  17 +#
  18 +# See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
  19 +RSpec.configure do |config|
  20 + # rspec-expectations config goes here. You can use an alternate
  21 + # assertion/expectation library such as wrong or the stdlib/minitest
  22 + # assertions if you prefer.
  23 + config.expect_with :rspec do |expectations|
  24 + # This option will default to `true` in RSpec 4. It makes the `description`
  25 + # and `failure_message` of custom matchers include text for helper methods
  26 + # defined using `chain`, e.g.:
  27 + # be_bigger_than(2).and_smaller_than(4).description
  28 + # # => "be bigger than 2 and smaller than 4"
  29 + # ...rather than:
  30 + # # => "be bigger than 2"
  31 + expectations.include_chain_clauses_in_custom_matcher_descriptions = true
  32 + end
  33 +
  34 + # rspec-mocks config goes here. You can use an alternate test double
  35 + # library (such as bogus or mocha) by changing the `mock_with` option here.
  36 + config.mock_with :rspec do |mocks|
  37 + # Prevents you from mocking or stubbing a method that does not exist on
  38 + # a real object. This is generally recommended, and will default to
  39 + # `true` in RSpec 4.
  40 + mocks.verify_partial_doubles = true
  41 + end
  42 +
  43 +# The settings below are suggested to provide a good initial experience
  44 +# with RSpec, but feel free to customize to your heart's content.
  45 +=begin
  46 + # These two settings work together to allow you to limit a spec run
  47 + # to individual examples or groups you care about by tagging them with
  48 + # `:focus` metadata. When nothing is tagged with `:focus`, all examples
  49 + # get run.
  50 + config.filter_run :focus
  51 + config.run_all_when_everything_filtered = true
  52 +
  53 + # Allows RSpec to persist some state between runs in order to support
  54 + # the `--only-failures` and `--next-failure` CLI options. We recommend
  55 + # you configure your source control system to ignore this file.
  56 + config.example_status_persistence_file_path = "spec/examples.txt"
  57 +
  58 + # Limits the available syntax to the non-monkey patched syntax that is
  59 + # recommended. For more details, see:
  60 + # - http://myronmars.to/n/dev-blog/2012/06/rspecs-new-expectation-syntax
  61 + # - http://www.teaisaweso.me/blog/2013/05/27/rspecs-new-message-expectation-syntax/
  62 + # - http://myronmars.to/n/dev-blog/2014/05/notable-changes-in-rspec-3#new__config_option_to_disable_rspeccore_monkey_patching
  63 + config.disable_monkey_patching!
  64 +
  65 + # Many RSpec users commonly either run the entire suite or an individual
  66 + # file, and it's useful to allow more verbose output when running an
  67 + # individual spec file.
  68 + if config.files_to_run.one?
  69 + # Use the documentation formatter for detailed output,
  70 + # unless a formatter has already been configured
  71 + # (e.g. via a command-line flag).
  72 + config.default_formatter = 'doc'
  73 + end
  74 +
  75 + # Print the 10 slowest examples and example groups at the
  76 + # end of the spec run, to help surface which specs are running
  77 + # particularly slow.
  78 + config.profile_examples = 10
  79 +
  80 + # Run specs in random order to surface order dependencies. If you find an
  81 + # order dependency and want to debug it, you can fix the order by providing
  82 + # the seed, which is printed after each run.
  83 + # --seed 1234
  84 + config.order = :random
  85 +
  86 + # Seed global randomization in this process using the `--seed` CLI option.
  87 + # Setting this allows you to use `--seed` to deterministically reproduce
  88 + # test failures related to randomization by passing the same `--seed` value
  89 + # as the one that triggered the failure.
  90 + Kernel.srand config.seed
  91 +=end
  92 +end
... ...