Commit 07b7133b755bde923745355c041960e779054230

Authored by Rodrigo Souto
1 parent ead9b1fb

Starting postgres search engine plugin

Showing 36 changed files with 2144 additions and 0 deletions   Show diff stats
plugins/pg_search/dependencies.rb 0 → 100644
... ... @@ -0,0 +1 @@
  1 +# Test if database is postgres
... ...
plugins/pg_search/lib/pg_search/.autotest 0 → 100644
... ... @@ -0,0 +1,5 @@
  1 +Autotest.add_hook :initialize do |at|
  2 + at.add_mapping(%r%^lib/.*\.rb$%, true) { |filename, _|
  3 + at.files_matching %r%^spec/.*_spec.rb$%
  4 + }
  5 +end
... ...
plugins/pg_search/lib/pg_search/.rvmrc 0 → 100644
... ... @@ -0,0 +1 @@
  1 +rvm use ree@pg_search
0 2 \ No newline at end of file
... ...
plugins/pg_search/lib/pg_search/CHANGELOG 0 → 100644
... ... @@ -0,0 +1,36 @@
  1 +### 0.2.1
  2 +
  3 + * Backport support for searching against tsvector columns (Kris Hicks)
  4 +
  5 +### 0.2
  6 +
  7 + * Set dictionary to :simple by default for :tsearch. Before it was unset,
  8 + which would fall back to PostgreSQL's default dictionary, usually
  9 + "english".
  10 +
  11 + * Fix a bug with search strings containing a colon ":"
  12 +
  13 + * Improve performance of :associated_against by only doing one INNER JOIN per
  14 + association
  15 +
  16 +### 0.1.1
  17 +
  18 + * Fix a bug with dmetaphone searches containing " w " (which dmetaphone maps
  19 + to an empty string)
  20 +
  21 +### 0.1
  22 +
  23 + * Change API to {:ignoring => :accents} from {:normalizing => :diacritics}
  24 +
  25 + * Improve documentation
  26 +
  27 + * Fix bug where :associated_against would not work without an :against
  28 + present
  29 +
  30 +### 0.0.2
  31 +
  32 + * Fix gem ownership.
  33 +
  34 +### 0.0.1
  35 +
  36 + * Initial release.
... ...
plugins/pg_search/lib/pg_search/Gemfile 0 → 100644
... ... @@ -0,0 +1,5 @@
  1 +puts <<-MESSAGE
  2 +This project uses multiple Gemfiles in subdirectories of ./gemfiles.
  3 +The rake tasks automatically install these bundles as necessary. See rake -T.
  4 +MESSAGE
  5 +exit 1
0 6 \ No newline at end of file
... ...
plugins/pg_search/lib/pg_search/LICENSE 0 → 100644
... ... @@ -0,0 +1,19 @@
  1 +Copyright (c) 2010-11 Case Commons, LLC
  2 +
  3 +Permission is hereby granted, free of charge, to any person obtaining a copy
  4 +of this software and associated documentation files (the "Software"), to deal
  5 +in the Software without restriction, including without limitation the rights
  6 +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  7 +copies of the Software, and to permit persons to whom the Software is
  8 +furnished to do so, subject to the following conditions:
  9 +
  10 +The above copyright notice and this permission notice shall be included in
  11 +all copies or substantial portions of the Software.
  12 +
  13 +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  14 +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  15 +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  16 +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  17 +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  18 +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  19 +THE SOFTWARE.
... ...
plugins/pg_search/lib/pg_search/README.rdoc 0 → 100644
... ... @@ -0,0 +1,351 @@
  1 += pg_search
  2 +
  3 +* http://github.com/casecommons/pg_search/
  4 +
  5 +== DESCRIPTION
  6 +
  7 +PgSearch builds named scopes that take advantage of PostgreSQL's full text search
  8 +
  9 +Read the blog post introducing PgSearch at http://bit.ly/pg_search
  10 +
  11 +== INSTALL
  12 +
  13 + gem install pg_search
  14 +
  15 +=== Rails 3
  16 +
  17 +In Gemfile
  18 +
  19 + gem 'pg_search'
  20 +
  21 +=== Rails 2
  22 +
  23 +In environment.rb
  24 +
  25 + config.gem 'pg_search'
  26 +
  27 +In Rakefile
  28 +
  29 + require 'rubygems'
  30 + require 'pg_search/tasks'
  31 +
  32 +== USAGE
  33 +
  34 +To add PgSearch to an ActiveRecord model, simply include the PgSearch module.
  35 +
  36 + class Shape < ActiveRecord::Base
  37 + include PgSearch
  38 + end
  39 +
  40 +=== pg_search_scope
  41 +
  42 +You can use pg_search_scope to build a search scope. The first parameter is a scope name, and the second parameter is an options hash. The only required option is :against, which tells pg_search_scope which column or columns to search against.
  43 +
  44 +==== Searching against one column
  45 +
  46 +To search against a column, pass a symbol as the :against option.
  47 +
  48 + class BlogPost < ActiveRecord::Base
  49 + include PgSearch
  50 + pg_search_scope :search_by_title, :against => :title
  51 + end
  52 +
  53 +We now have an ActiveRecord scope named search_by_title on our BlogPost model. It takes one parameter, a search query string.
  54 +
  55 + BlogPost.create!(:title => "Recent Developments in the World of Pastrami")
  56 + BlogPost.create!(:title => "Prosciutto and You: A Retrospective")
  57 + BlogPost.search_by_title("pastrami") # => [#<BlogPost id: 2, title: "Recent Developments in the World of Pastrami">]
  58 +
  59 +==== Searching against multiple columns
  60 +
  61 +Just pass an Array if you'd like to search more than one column.
  62 +
  63 + class Person < ActiveRecord::Base
  64 + include PgSearch
  65 + pg_search_scope :search_by_full_name, :against => [:first_name, :last_name]
  66 + end
  67 +
  68 +Now our search query can match either or both of the columns.
  69 +
  70 + person_1 = Person.create!(:first_name => "Grant", :last_name => "Hill")
  71 + person_2 = Person.create!(:first_name => "Hugh", :last_name => "Grant")
  72 +
  73 + Person.search_by_full_name("Grant") # => [person_1, person_2]
  74 + Person.search_by_full_name("Grant Hill") # => [person_1]
  75 +
  76 +==== Dynamic search scopes
  77 +
  78 +Just like with Active Record named scopes, you can pass in a Proc object that returns a hash of options. For instance, the following scope takes a parameter that dynamically chooses which column to search against.
  79 +
  80 +Important: The returned hash must include a :query key. Its value does not necessary have to be dynamic. You could choose to hard-code it to a specific value if you wanted.
  81 +
  82 + class Person < ActiveRecord::Base
  83 + include PgSearch
  84 + pg_search_scope :search_by_name, lambda do |name_part, query|
  85 + raise ArgumentError unless [:first, :last].include?(name_part)
  86 + {
  87 + :against => name_part,
  88 + :query => query
  89 + }
  90 + end
  91 + end
  92 +
  93 + person_1 = Person.create!(:first_name => "Grant", :last_name => "Hill")
  94 + person_2 = Person.create!(:first_name => "Hugh", :last_name => "Grant")
  95 +
  96 + Person.search_by_name :first, "Grant" # => [person_1]
  97 + Person.search_by_name :last, "Grant" # => [person_2]
  98 +
  99 +==== Searching through associations
  100 +
  101 +You can pass a Hash into the :associated_against option to search columns on other models. The keys are the names of the associations and the value works just like an :against option for the other model.
  102 +
  103 + class Cracker < ActiveRecord::Base
  104 + has_many :cheeses
  105 + end
  106 +
  107 + class Cheese < ActiveRecord::Base
  108 + end
  109 +
  110 + class Salami < ActiveRecord::Base
  111 + include PgSearch
  112 +
  113 + belongs_to :cracker
  114 + has_many :cheeses, :through => :cracker
  115 +
  116 + pg_search_scope :tasty_search, :associated_against => {
  117 + :cheeses => [:kind, :brand],
  118 + :cracker => :kind
  119 + }
  120 + end
  121 +
  122 + salami_1 = Salami.create!
  123 + salami_2 = Salami.create!
  124 + salami_3 = Salami.create!
  125 +
  126 + limburger = Cheese.create!(:kind => "Limburger")
  127 + brie = Cheese.create!(:kind => "Brie")
  128 + pepper_jack = Cheese.create!(:kind => "Pepper Jack")
  129 +
  130 + Cracker.create!(:kind => "Black Pepper", :cheeses => [brie], :salami => salami_1)
  131 + Cracker.create!(:kind => "Ritz", :cheeses => [limburger, pepper_jack], :salami => salami_2)
  132 + Cracker.create!(:kind => "Graham", :cheeses => [limburger], :salami => salami_3)
  133 +
  134 + Salami.tasty_search("pepper") # => [salami_1, salami_2]
  135 +
  136 +=== Searching using different search features
  137 +
  138 +By default, pg_search_scope uses the built-in {PostgreSQL text search}[http://www.postgresql.org/docs/current/static/textsearch-intro.html]. If you pass the :features option to pg_search_scope, you can choose alternative search techniques.
  139 +
  140 + class Beer < ActiveRecord::Base
  141 + include PgSearch
  142 + pg_search_scope :against => :name, :features => [:tsearch, :trigram, :dmetaphone]
  143 + end
  144 +
  145 +The currently implemented features are
  146 +
  147 +* :tsearch - {Full text search}[http://www.postgresql.org/docs/current/static/textsearch-intro.html] (built-in with 8.3 and later, available as a contrib package for some earlier versions)
  148 +* :trigram - {Trigram search}[http://www.postgresql.org/docs/current/static/pgtrgm.html], which requires the trigram contrib package
  149 +* :dmetaphone - {Double Metaphone search}[http://www.postgresql.org/docs/9.0/static/fuzzystrmatch.html#AEN120188], which requires the fuzzystrmatch contrib package
  150 +
  151 +==== :tsearch (Full Text Search)
  152 +
  153 +PostgreSQL's built-in full text search supports weighting, prefix searches, and stemming in multiple languages.
  154 +
  155 +===== Weighting
  156 +Each searchable column can be given a weight of "A", "B", "C", or "D". Columns with earlier letters are weighted higher than those with later letters. So, in the following example, the title is the most important, followed by the subtitle, and finally the content.
  157 +
  158 + class NewsArticle < ActiveRecord::Base
  159 + include PgSearch
  160 + pg_search_scope :against => {
  161 + :title => 'A',
  162 + :subtitle => 'B',
  163 + :content => 'C'
  164 + }
  165 + end
  166 +
  167 +You can also pass the weights in as an array of arrays, or any other structure that responds to #each and yields either a single symbol or a symbol and a weight. If you omit the weight, a default will be used.
  168 +
  169 + class NewsArticle < ActiveRecord::Base
  170 + include PgSearch
  171 + pg_search_scope :against => [
  172 + [:title, 'A'],
  173 + [:subtitle, 'B'],
  174 + [:content, 'C']
  175 + ]
  176 + end
  177 +
  178 + class NewsArticle < ActiveRecord::Base
  179 + include PgSearch
  180 + pg_search_scope :against => [
  181 + [:title, 'A'],
  182 + {:subtitle => 'B'},
  183 + :content
  184 + ]
  185 + end
  186 +
  187 +===== :prefix
  188 +
  189 +PostgreSQL's full text search matches on whole words by default. If you want to search for partial words, however, you can set :prefix to true. Since this is a :tsearch-specific option, you should pass it to :tsearch directly, as shown in the following example.
  190 +
  191 + class Superhero < ActiveRecord::Base
  192 + include PgSearch
  193 + pg_search_scope :whose_name_starts_with,
  194 + :against => :name,
  195 + :using => {
  196 + :tsearch => {:prefix => true}
  197 + }
  198 + end
  199 +
  200 + batman = Superhero.create :name => 'Batman'
  201 + batgirl = Superhero.create :name => 'Batgirl'
  202 + robin = Superhero.create :name => 'Robin'
  203 +
  204 + Superhero.whose_name_starts_with("Bat") # => [batman, batgirl]
  205 +
  206 +===== :dictionary
  207 +
  208 +PostgreSQL full text search also support multiple dictionaries for stemming. You can learn more about how dictionaries work by reading the {PostgreSQL documention}[http://www.postgresql.org/docs/current/static/textsearch-dictionaries.html]. If you use one of the language dictionaries, such as "english", then variants of words (e.g. "jumping" and "jumped") will match each other. If you don't want stemming, you should pick the "simple" dictionary which does not do any stemming. If you don't specify a dictionary, the "simple" dictionary will be used.
  209 +
  210 + class BoringTweet < ActiveRecord::Base
  211 + include PgSearch
  212 + pg_search_scope :kinda_matching,
  213 + :against => :text,
  214 + :using => {
  215 + :tsearch => {:dictionary => "english"}
  216 + }
  217 + pg_search_scope :literally_matching,
  218 + :against => :text,
  219 + :using => {
  220 + :tsearch => {:dictionary => "simple"}
  221 + }
  222 + end
  223 +
  224 + sleepy = BoringTweet.create! :text => "I snoozed my alarm for fourteen hours today. I bet I can beat that tomorrow! #sleepy"
  225 + sleeping = BoringTweet.create! :text => "You know what I like? Sleeping. That's what. #enjoyment"
  226 + sleeper = BoringTweet.create! :text => "Have you seen Woody Allen's movie entitled Sleeper? Me neither. #boycott"
  227 +
  228 + BoringTweet.kinda_matching("sleeping") # => [sleepy, sleeping, sleeper]
  229 + BoringTweet.literally_matching("sleeping") # => [sleeping]
  230 +
  231 +==== :dmetaphone (Double Metaphone soundalike search)
  232 +
  233 +{Double Metaphone}[http://en.wikipedia.org/wiki/Double_Metaphone] is an algorithm for matching words that sound alike even if they are spelled very differently. For example, "Geoff" and "Jeff" sound identical and thus match. Currently, this is not a true double-metaphone, as only the first metaphone is used for searching.
  234 +
  235 +Double Metaphone support is currently available as part of the {fuzzystrmatch contrib package}[http://www.postgresql.org/docs/current/static/fuzzystrmatch.html] that must be installed before this feature can be used. In addition to the contrib package, you must install a utility function into your database. To generate a migration for this, add the following line to your Rakefile:
  236 +
  237 + include "pg_search/tasks"
  238 +
  239 +and then run:
  240 +
  241 + $ rake pg_search:migration:dmetaphone
  242 +
  243 +The following example shows how to use :dmetaphone.
  244 +
  245 + class Word < ActiveRecord::Base
  246 + include PgSearch
  247 + pg_search_scope :that_sounds_like,
  248 + :against => :spelling,
  249 + :using => :dmetaphone
  250 + end
  251 +
  252 + four = Word.create! :spelling => 'four'
  253 + far = Word.create! :spelling => 'far'
  254 + fur = Word.create! :spelling => 'fur'
  255 + five = Word.create! :spelling => 'five'
  256 +
  257 + Word.that_sounds_like("fir") # => [four, far, fur]
  258 +
  259 +==== :trigram (Trigram search)
  260 +
  261 +Trigram search works by counting how many three-letter substrings (or "trigrams") match between the query and the text. For example, the string "Lorem ipsum" can be split into the following trigrams:
  262 +
  263 + [" Lo", "Lor", "ore", "rem", "em ", "m i", " ip", "ips", "psu", "sum", "um ", "m "]
  264 +
  265 +Trigram search has some ability to work even with typos and misspellings in the query or text.
  266 +
  267 +Trigram support is currently available as part of the {pg_trgm contrib package}[http://www.postgresql.org/docs/current/static/pgtrgm.html] that must be installed before this feature can be used.
  268 +
  269 +
  270 + class Website < ActiveRecord::Base
  271 + include PgSearch
  272 + pg_search_scope :kinda_spelled_like,
  273 + :against => :name,
  274 + :using => :trigram
  275 + end
  276 +
  277 + yahooo = Website.create! :name => "Yahooo!"
  278 + yohoo = Website.create! :name => "Yohoo!"
  279 + gogle = Website.create! :name => "Gogle"
  280 + facebook = Website.create! :name => "Facebook"
  281 +
  282 + Website.kinda_spelled_like("Yahoo!") # => [yahooo, yohoo]
  283 +
  284 +=== Ignoring accent marks
  285 +
  286 +Most of the time you will want to ignore accent marks when searching. This makes it possible to find words like "piñata" when searching with the query "pinata". If you set a pg_search_scope to ignore accents, it will ignore accents in both the searchable text and the query terms.
  287 +
  288 +Ignoring accents uses the {unaccent contrib package}[http://www.postgresql.org/docs/current/static/unaccent.html] that must be installed before this feature can be used.
  289 +
  290 +
  291 + class SpanishQuestion < ActiveRecord::Base
  292 + include PgSearch
  293 + pg_search_scope :gringo_search,
  294 + :against => :word,
  295 + :ignoring => :accents
  296 + end
  297 +
  298 + what = SpanishQuestion.create(:word => "Qué")
  299 + how_many = SpanishQuestion.create(:word => "Cuánto")
  300 + how = SpanishQuestion.create(:word => "Cómo")
  301 +
  302 + SpanishQuestion.gringo_search("Que") # => [what]
  303 + SpanishQuestion.gringo_search("Cüåñtô") # => [how_many]
  304 +
  305 +=== Using tsvector columns
  306 +
  307 +PostgreSQL allows you the ability to search against a column with type tsvector instead of using an expression; this speeds up searching dramatically as it offloads creation of the tsvector that the tsquery is evaluated against.
  308 +
  309 +To use this functionality you'll need to do a few things:
  310 +
  311 +* Create a column of type tsvector that you'd like to search against. If you want to search using multiple search methods, for example tsearch and dmetaphone, you'll need a column for each.
  312 +* Create a trigger function that will update the column(s) using the expression appropriate for that type of search. See: http://www.postgresql.org/docs/current/static/textsearch-features.html#TEXTSEARCH-UPDATE-TRIGGERS
  313 +* Should you have any pre-existing data in the table, update the newly-created tsvector columns with the expression that your trigger function uses.
  314 +* Add the option to pg_search_scope, e.g:
  315 +
  316 + pg_search_scope :fast_content_search,
  317 + :against => :content,
  318 + :using => {
  319 + dmetaphone: {
  320 + tsvector_column: 'tsvector_content_dmetaphone'
  321 + },
  322 + tsearch: {
  323 + dictionary: 'english',
  324 + tsvector_column: 'tsvector_content_tsearch'
  325 + }
  326 + trigram: {} # trigram does not use tsvectors
  327 + }
  328 +
  329 +Please note that the :against column is only used when the tsvector_column is not present for the search type.
  330 +
  331 +== REQUIREMENTS
  332 +
  333 +* ActiveRecord 2 or 3
  334 +* Postgresql
  335 +* Postgresql contrib modules for certain features
  336 +
  337 +== ATTRIBUTIONS
  338 +
  339 +PgSearch would not have been possible without inspiration from
  340 +{texticle}[https://github.com/tenderlove/texticle]. Thanks to
  341 +{Aaron Patterson}[http://tenderlovemaking.com/]!
  342 +
  343 +== CONTRIBUTIONS AND FEEDBACK
  344 +
  345 +Welcomed! Feel free to join and contribute to our {public Pivotal Tracker project}[https://www.pivotaltracker.com/projects/228645] where we manage new feature ideas and bugs.
  346 +
  347 +We also have a {Google Group}[http://groups.google.com/group/casecommons-dev] for discussing pg_search and other Case Commons open source projects.
  348 +
  349 +== LICENSE
  350 +
  351 +MIT
... ...
plugins/pg_search/lib/pg_search/Rakefile 0 → 100644
... ... @@ -0,0 +1,44 @@
  1 +require 'bundler'
  2 +Bundler::GemHelper.install_tasks
  3 +
  4 +task :default => :spec
  5 +
  6 +environments = %w[rails2 rails3]
  7 +major, minor, revision = RUBY_VERSION.split(".").map{|str| str.to_i }
  8 +
  9 +in_environment = lambda do |environment, command|
  10 + sh %Q{export BUNDLE_GEMFILE="gemfiles/#{environment}/Gemfile"; bundle update && bundle exec #{command}}
  11 +end
  12 +
  13 +in_all_environments = lambda do |command|
  14 + environments.each do |environment|
  15 + next if environment == "rails2" && major == 1 && minor > 8
  16 + puts "\n---#{environment}---\n"
  17 + in_environment.call(environment, command)
  18 + end
  19 +end
  20 +
  21 +desc "Run all specs against ActiveRecord 2 and 3"
  22 +task "spec" do
  23 + in_all_environments.call('rspec spec')
  24 +end
  25 +
  26 +task "doc" do
  27 + in_environment.call("rails3", "rspec --format d spec")
  28 +end
  29 +
  30 +namespace "autotest" do
  31 + environments.each do |environment|
  32 + desc "Run autotest in #{environment}"
  33 + task environment do
  34 + in_environment.call(environment, 'autotest -s rspec2')
  35 + end
  36 + end
  37 +end
  38 +
  39 +namespace "doc" do
  40 + desc "Generate README and preview in browser"
  41 + task "readme" do
  42 + sh "rdoc -c utf8 README.rdoc && open doc/files/README_rdoc.html"
  43 + end
  44 +end
... ...
plugins/pg_search/lib/pg_search/TODO 0 → 100644
... ... @@ -0,0 +1,10 @@
  1 + * Railtie for rake tasks
  2 + * Tracker project
  3 + * Exceptions for missing trigram, dmetaphone, etc. support
  4 + * LIKE search
  5 + * accept a block and pass it to the underlying scope
  6 + * ability to search again a tsvector column
  7 + * Generate indexes
  8 + * Allow for chaining multiple pg_search scopes
  9 +
  10 +Also see https://github.com/Casecommons/pg_search/issues
... ...
plugins/pg_search/lib/pg_search/defaut_search.rb 0 → 100644
... ... @@ -0,0 +1,8 @@
  1 +searchables = %w[ article comment qualifier article national_region certifier profile license scrap category ]
  2 +searchables.each { |searchable| require_dependency searchable }
  3 +klasses = searchables.map {|searchable| searchable.camelize.constantize}
  4 +
  5 +klasses.module_eval do
  6 + include PgSearch
  7 + pg_search_scope :pg_search_plugin_search, :against => SEARCHABLE_FIELDS
  8 +end
... ...
plugins/pg_search/lib/pg_search/gemfiles/Gemfile.common 0 → 100644
... ... @@ -0,0 +1,9 @@
  1 +source "http://rubygems.org"
  2 +
  3 +gem "pg"
  4 +
  5 +group :test do
  6 + gem "rspec", ">=2.4"
  7 + gem "autotest"
  8 + gem "with_model"
  9 +end
... ...
plugins/pg_search/lib/pg_search/gemfiles/rails2/Gemfile 0 → 100644
... ... @@ -0,0 +1,4 @@
  1 +filename = File.join(File.dirname(__FILE__), '..', 'Gemfile.common')
  2 +eval(File.read(filename), binding, filename, 1)
  3 +
  4 +gem "activerecord", "~>2.3.5"
... ...
plugins/pg_search/lib/pg_search/gemfiles/rails3/Gemfile 0 → 100644
... ... @@ -0,0 +1,4 @@
  1 +filename = File.join(File.dirname(__FILE__), '..', 'Gemfile.common')
  2 +eval(File.read(filename), binding, filename, 1)
  3 +
  4 +gem "activerecord", "~>3.0.1"
... ...
plugins/pg_search/lib/pg_search/lib/pg_search.rb 0 → 100644
... ... @@ -0,0 +1,32 @@
  1 +require "active_record"
  2 +require "pg_search/configuration"
  3 +require "pg_search/features"
  4 +require "pg_search/normalizer"
  5 +require "pg_search/scope"
  6 +require "pg_search/scope_options"
  7 +require "pg_search/version"
  8 +#require "pg_search/railtie" if defined?(Rails) && defined?(Rails::Railtie)
  9 +
  10 +module PgSearch
  11 + def self.included(base)
  12 + base.send(:extend, ClassMethods)
  13 + end
  14 +
  15 + module ClassMethods
  16 + def pg_search_scope(name, options)
  17 + scope = PgSearch::Scope.new(name, self, options)
  18 + scope_method =
  19 + if respond_to?(:scope) && !protected_methods.include?('scope')
  20 + :scope # ActiveRecord 3.x
  21 + else
  22 + :named_scope # ActiveRecord 2.x
  23 + end
  24 +
  25 + send(scope_method, name, scope.to_proc)
  26 + end
  27 + end
  28 +
  29 + def rank
  30 + attributes['pg_search_rank'].to_f
  31 + end
  32 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/configuration.rb 0 → 100644
... ... @@ -0,0 +1,80 @@
  1 +require "pg_search/configuration/association"
  2 +require "pg_search/configuration/column"
  3 +
  4 +module PgSearch
  5 + class Configuration
  6 + def initialize(options, model)
  7 + options = options.reverse_merge(default_options)
  8 + assert_valid_options(options)
  9 + @options = options
  10 + @model = model
  11 + end
  12 +
  13 + def columns
  14 + regular_columns + associated_columns
  15 + end
  16 +
  17 + def regular_columns
  18 + return [] unless @options[:against]
  19 + Array(@options[:against]).map do |column_name, weight|
  20 + Column.new(column_name, weight, @model)
  21 + end
  22 + end
  23 +
  24 + def associations
  25 + return [] unless @options[:associated_against]
  26 + @options[:associated_against].map do |association, column_names|
  27 + association = Association.new(@model, association, column_names)
  28 + association
  29 + end.flatten
  30 + end
  31 +
  32 + def associated_columns
  33 + associations.map(&:columns).flatten
  34 + end
  35 +
  36 + def query
  37 + @options[:query].to_s
  38 + end
  39 +
  40 + def ignore
  41 + Array.wrap(@options[:ignoring])
  42 + end
  43 +
  44 + def ranking_sql
  45 + @options[:ranked_by]
  46 + end
  47 +
  48 + def features
  49 + Array(@options[:using])
  50 + end
  51 +
  52 + private
  53 +
  54 + def default_options
  55 + {:using => :tsearch}
  56 + end
  57 +
  58 + def assert_valid_options(options)
  59 + valid_keys = [:against, :ranked_by, :ignoring, :using, :query, :associated_against]
  60 + valid_values = {
  61 + :ignoring => [:accents]
  62 + }
  63 +
  64 + unless options[:against] || options[:associated_against]
  65 + raise ArgumentError, "the search scope #{@name} must have :against#{" or :associated_against" if defined?(ActiveRecord::Relation)} in its options"
  66 + end
  67 + raise ArgumentError, ":associated_against requires ActiveRecord 3 or later" if options[:associated_against] && !defined?(ActiveRecord::Relation)
  68 +
  69 + options.assert_valid_keys(valid_keys)
  70 +
  71 + valid_values.each do |key, values_for_key|
  72 + Array.wrap(options[key]).each do |value|
  73 + unless values_for_key.include?(value)
  74 + raise ArgumentError, ":#{key} cannot accept #{value}"
  75 + end
  76 + end
  77 + end
  78 + end
  79 + end
  80 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/configuration/association.rb 0 → 100644
... ... @@ -0,0 +1,34 @@
  1 +require "digest"
  2 +
  3 +module PgSearch
  4 + class Configuration
  5 + class Association
  6 + attr_reader :columns
  7 +
  8 + def initialize(model, name, column_names)
  9 + @model = model
  10 + @name = name
  11 + @columns = Array(column_names).map do |column_name, weight|
  12 + Column.new(column_name, weight, @model, self)
  13 + end
  14 + end
  15 +
  16 + def table_name
  17 + @model.reflect_on_association(@name).table_name
  18 + end
  19 +
  20 + def join(primary_key)
  21 + selects = columns.map do |column|
  22 + "string_agg(#{column.full_name}, ' ') AS #{column.alias}"
  23 + end.join(", ")
  24 + relation = @model.joins(@name).select("#{primary_key} AS id, #{selects}").group(primary_key)
  25 + "LEFT OUTER JOIN (#{relation.to_sql}) #{subselect_alias} ON #{subselect_alias}.id = #{primary_key}"
  26 + end
  27 +
  28 + def subselect_alias
  29 + subselect_name = ["pg_search", table_name, @name, "subselect"].compact.join('_')
  30 + "pg_search_#{Digest::SHA2.hexdigest(subselect_name)}"
  31 + end
  32 + end
  33 + end
  34 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/configuration/column.rb 0 → 100644
... ... @@ -0,0 +1,42 @@
  1 +require 'digest'
  2 +
  3 +module PgSearch
  4 + class Configuration
  5 + class Column
  6 + attr_reader :weight, :association
  7 +
  8 + def initialize(column_name, weight, model, association = nil)
  9 + @column_name = column_name.to_s
  10 + @weight = weight
  11 + @model = model
  12 + @association = association
  13 + end
  14 +
  15 + def table
  16 + foreign? ? @association.table_name : @model.table_name
  17 + end
  18 +
  19 + def full_name
  20 + "#{@model.connection.quote_table_name(table)}.#{@model.connection.quote_column_name(@column_name)}"
  21 + end
  22 +
  23 + def to_sql
  24 + name = if foreign?
  25 + "#{@association.subselect_alias}.#{self.alias}"
  26 + else
  27 + full_name
  28 + end
  29 + "coalesce(#{name}, '')"
  30 + end
  31 +
  32 + def foreign?
  33 + @association.present?
  34 + end
  35 +
  36 + def alias
  37 + name = [association.subselect_alias, @column_name].compact.join('_')
  38 + "pg_search_#{Digest::SHA2.hexdigest(name)}"
  39 + end
  40 + end
  41 + end
  42 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/features.rb 0 → 100644
... ... @@ -0,0 +1,7 @@
  1 +module PgSearch
  2 + module Features
  3 + end
  4 +end
  5 +require 'pg_search/features/dmetaphone'
  6 +require 'pg_search/features/trigram'
  7 +require 'pg_search/features/tsearch'
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/features/dmetaphone.rb 0 → 100644
... ... @@ -0,0 +1,28 @@
  1 +require "active_support/core_ext/module/delegation"
  2 +
  3 +module PgSearch
  4 + module Features
  5 + class DMetaphone
  6 + delegate :conditions, :rank, :to => :'@tsearch'
  7 +
  8 + # config is temporary as we refactor
  9 + def initialize(query, options, config, model, normalizer)
  10 + dmetaphone_normalizer = Normalizer.new(normalizer)
  11 + options = (options || {}).merge(:dictionary => 'simple')
  12 + @tsearch = TSearch.new(query, options, config, model, dmetaphone_normalizer)
  13 + end
  14 +
  15 + # Decorates a normalizer with dmetaphone processing.
  16 + class Normalizer
  17 + def initialize(normalizer_to_wrap)
  18 + @decorated_normalizer = normalizer_to_wrap
  19 + end
  20 +
  21 + def add_normalization(original_sql)
  22 + otherwise_normalized_sql = @decorated_normalizer.add_normalization(original_sql)
  23 + "pg_search_dmetaphone(#{otherwise_normalized_sql})"
  24 + end
  25 + end
  26 + end
  27 + end
  28 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/features/trigram.rb 0 → 100644
... ... @@ -0,0 +1,29 @@
  1 +require "active_support/core_ext/module/delegation"
  2 +
  3 +module PgSearch
  4 + module Features
  5 + class Trigram
  6 + def initialize(query, options, columns, model, normalizer)
  7 + @query = query
  8 + @options = options
  9 + @columns = columns
  10 + @model = model
  11 + @normalizer = normalizer
  12 + end
  13 +
  14 + def conditions
  15 + ["(#{@normalizer.add_normalization(document)}) % #{@normalizer.add_normalization(":query")}", {:query => @query}]
  16 + end
  17 +
  18 + def rank
  19 + ["similarity((#{@normalizer.add_normalization(document)}), #{@normalizer.add_normalization(":query")})", {:query => @query}]
  20 + end
  21 +
  22 + private
  23 +
  24 + def document
  25 + @columns.map { |column| column.to_sql }.join(" || ' ' || ")
  26 + end
  27 + end
  28 + end
  29 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/features/tsearch.rb 0 → 100644
... ... @@ -0,0 +1,72 @@
  1 +require "active_support/core_ext/module/delegation"
  2 +
  3 +module PgSearch
  4 + module Features
  5 + class TSearch
  6 + delegate :connection, :quoted_table_name, :to => :'@model'
  7 +
  8 + def initialize(query, options, columns, model, normalizer)
  9 + @query = query
  10 + @options = options || {}
  11 + @model = model
  12 + @columns = columns
  13 + @normalizer = normalizer
  14 + end
  15 +
  16 + def conditions
  17 + ["(#{tsdocument}) @@ (#{tsquery})", interpolations]
  18 + end
  19 +
  20 + def rank
  21 + tsearch_rank
  22 + end
  23 +
  24 + private
  25 +
  26 + def interpolations
  27 + {:query => @query.to_s, :dictionary => dictionary.to_s}
  28 + end
  29 +
  30 + def document
  31 + @columns.map { |column| column.to_sql }.join(" || ' ' || ")
  32 + end
  33 +
  34 + def tsquery
  35 + return "''" if @query.blank?
  36 +
  37 + @query.split(" ").compact.map do |term|
  38 + sanitized_term = term.gsub(/['?\-\\:]/, " ")
  39 +
  40 + term_sql = @normalizer.add_normalization(connection.quote(sanitized_term))
  41 +
  42 + # After this, the SQL expression evaluates to a string containing the term surrounded by single-quotes.
  43 + tsquery_sql = "#{connection.quote("' ")} || #{term_sql} || #{connection.quote(" '")}"
  44 +
  45 + # Add tsearch prefix operator if we're using a prefix search.
  46 + tsquery_sql = "#{tsquery_sql} || #{connection.quote(':*')}" if @options[:prefix]
  47 +
  48 + "to_tsquery(:dictionary, #{tsquery_sql})"
  49 + end.join(" && ")
  50 + end
  51 +
  52 + def tsdocument
  53 + if @options[:tsvector_column]
  54 + @options[:tsvector_column].to_s
  55 + else
  56 + @columns.map do |search_column|
  57 + tsvector = "to_tsvector(:dictionary, #{@normalizer.add_normalization(search_column.to_sql)})"
  58 + search_column.weight.nil? ? tsvector : "setweight(#{tsvector}, #{connection.quote(search_column.weight)})"
  59 + end.join(" || ")
  60 + end
  61 + end
  62 +
  63 + def tsearch_rank
  64 + ["ts_rank((#{tsdocument}), (#{tsquery}))", interpolations]
  65 + end
  66 +
  67 + def dictionary
  68 + @options[:dictionary] || :simple
  69 + end
  70 + end
  71 + end
  72 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/normalizer.rb 0 → 100644
... ... @@ -0,0 +1,13 @@
  1 +module PgSearch
  2 + class Normalizer
  3 + def initialize(config)
  4 + @config = config
  5 + end
  6 +
  7 + def add_normalization(original_sql)
  8 + normalized_sql = original_sql
  9 + normalized_sql = "unaccent(#{normalized_sql})" if @config.ignore.include?(:accents)
  10 + normalized_sql
  11 + end
  12 + end
  13 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/railtie.rb 0 → 100644
... ... @@ -0,0 +1,11 @@
  1 +require 'pg_search'
  2 +#require 'rails'
  3 +
  4 +module PgSearch
  5 + class Railtie < Rails::Railtie
  6 + rake_tasks do
  7 + raise
  8 + load "pg_search/tasks.rb"
  9 + end
  10 + end
  11 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/scope.rb 0 → 100644
... ... @@ -0,0 +1,31 @@
  1 +module PgSearch
  2 + class Scope
  3 + def initialize(name, model, scope_options_or_proc)
  4 + @name = name
  5 + @model = model
  6 + @options_proc = build_options_proc(scope_options_or_proc)
  7 + end
  8 +
  9 + def to_proc
  10 + lambda { |*args|
  11 + config = Configuration.new(@options_proc.call(*args), @model)
  12 + ScopeOptions.new(@name, @model, config).to_hash
  13 + }
  14 + end
  15 +
  16 + private
  17 +
  18 + def build_options_proc(scope_options_or_proc)
  19 + case scope_options_or_proc
  20 + when Proc
  21 + scope_options_or_proc
  22 + when Hash
  23 + lambda { |query|
  24 + scope_options_or_proc.reverse_merge(:query => query)
  25 + }
  26 + else
  27 + raise ArgumentError, "A PgSearch scope expects a Proc or Hash for its options"
  28 + end
  29 + end
  30 + end
  31 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/scope_options.rb 0 → 100644
... ... @@ -0,0 +1,73 @@
  1 +require "active_support/core_ext/module/delegation"
  2 +
  3 +module PgSearch
  4 + class ScopeOptions
  5 + attr_reader :model
  6 +
  7 + delegate :connection, :quoted_table_name, :sanitize_sql_array, :to => :model
  8 +
  9 + def initialize(name, model, config)
  10 + @name = name
  11 + @model = model
  12 + @config = config
  13 +
  14 + @feature_options = @config.features.inject({}) do |features_hash, (feature_name, feature_options)|
  15 + features_hash.merge(
  16 + feature_name => feature_options
  17 + )
  18 + end
  19 + @feature_names = @config.features.map { |feature_name, feature_options| feature_name }
  20 + end
  21 +
  22 + def to_hash
  23 + {
  24 + :select => "#{quoted_table_name}.*, (#{rank}) AS pg_search_rank",
  25 + :conditions => conditions,
  26 + :order => "pg_search_rank DESC, #{primary_key} ASC",
  27 + :joins => joins
  28 + }
  29 + end
  30 +
  31 + private
  32 +
  33 + def conditions
  34 + @feature_names.map { |feature_name| "(#{sanitize_sql_array(feature_for(feature_name).conditions)})" }.join(" OR ")
  35 + end
  36 +
  37 + def primary_key
  38 + "#{quoted_table_name}.#{connection.quote_column_name(model.primary_key)}"
  39 + end
  40 +
  41 + def joins
  42 + @config.associations.map do |association|
  43 + association.join(primary_key)
  44 + end.join(' ')
  45 + end
  46 +
  47 + def feature_for(feature_name)
  48 + feature_name = feature_name.to_sym
  49 +
  50 + feature_class = {
  51 + :dmetaphone => Features::DMetaphone,
  52 + :tsearch => Features::TSearch,
  53 + :trigram => Features::Trigram
  54 + }[feature_name]
  55 +
  56 + raise ArgumentError.new("Unknown feature: #{feature_name}") unless feature_class
  57 +
  58 + normalizer = Normalizer.new(@config)
  59 +
  60 + feature_class.new(@config.query, @feature_options[feature_name], @config.columns, @model, normalizer)
  61 + end
  62 +
  63 + def tsearch_rank
  64 + sanitize_sql_array(@feature_names[Features::TSearch].rank)
  65 + end
  66 +
  67 + def rank
  68 + (@config.ranking_sql || ":tsearch").gsub(/:(\w*)/) do
  69 + sanitize_sql_array(feature_for($1).rank)
  70 + end
  71 + end
  72 + end
  73 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/tasks.rb 0 → 100644
... ... @@ -0,0 +1,37 @@
  1 +require 'rake'
  2 +require 'pg_search'
  3 +
  4 +namespace :pg_search do
  5 + namespace :migration do
  6 + desc "Generate migration to add support functions for :dmetaphone"
  7 + task :dmetaphone do
  8 + now = Time.now.utc
  9 + filename = "#{now.strftime('%Y%m%d%H%M%S')}_add_pg_search_dmetaphone_support_functions.rb"
  10 +
  11 + dmetaphone_sql = File.read(File.join(File.dirname(__FILE__), '..', '..', 'sql', 'dmetaphone.sql')).chomp
  12 + uninstall_dmetaphone_sql = File.read(File.join(File.dirname(__FILE__), '..', '..', 'sql', 'uninstall_dmetaphone.sql')).chomp
  13 +
  14 + File.open(Rails.root + 'db' + 'migrate' + filename, 'wb') do |migration_file|
  15 + migration_file.puts <<-RUBY
  16 +class AddPgSearchDmetaphoneSupportFunctions < ActiveRecord::Migration
  17 + def self.up
  18 + say_with_time("Adding support functions for pg_search :dmetaphone") do
  19 + ActiveRecord::Base.connection.execute(<<-SQL)
  20 + #{dmetaphone_sql}
  21 + SQL
  22 + end
  23 + end
  24 +
  25 + def self.down
  26 + say_with_time("Dropping support functions for pg_search :dmetaphone") do
  27 + ActiveRecord::Base.connection.execute(<<-SQL)
  28 + #{uninstall_dmetaphone_sql}
  29 + SQL
  30 + end
  31 + end
  32 +end
  33 + RUBY
  34 + end
  35 + end
  36 + end
  37 +end
... ...
plugins/pg_search/lib/pg_search/lib/pg_search/version.rb 0 → 100644
... ... @@ -0,0 +1,3 @@
  1 +module PgSearch
  2 + VERSION = "0.2.1"
  3 +end
... ...
plugins/pg_search/lib/pg_search/pg_search.gemspec 0 → 100644
... ... @@ -0,0 +1,19 @@
  1 +# -*- encoding: utf-8 -*-
  2 +$:.push File.expand_path("../lib", __FILE__)
  3 +require "pg_search/version"
  4 +
  5 +Gem::Specification.new do |s|
  6 + s.name = "pg_search"
  7 + s.version = PgSearch::VERSION
  8 + s.platform = Gem::Platform::RUBY
  9 + s.authors = ["Case Commons, LLC"]
  10 + s.email = ["casecommons-dev@googlegroups.com"]
  11 + s.homepage = "https://github.com/Casecommons/pg_search"
  12 + s.summary = %q{PgSearch builds ActiveRecord named scopes that take advantage of PostgreSQL's full text search}
  13 + s.description = %q{PgSearch builds ActiveRecord named scopes that take advantage of PostgreSQL's full text search}
  14 +
  15 + s.files = `git ls-files`.split("\n")
  16 + s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
  17 + s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
  18 + s.require_paths = ["lib"]
  19 +end
... ...
plugins/pg_search/lib/pg_search/script/setup-contrib 0 → 100755
... ... @@ -0,0 +1,12 @@
  1 +#!/bin/sh
  2 +POSTGRESQL_VERSION=`pg_config --version | awk '{print $2}'`
  3 +
  4 +cd /tmp
  5 +test -e /tmp/postgresql-$POSTGRESQL_VERSION.tar.bz2 || wget http://ftp9.us.postgresql.org/pub/mirrors/postgresql/source/v$POSTGRESQL_VERSION/postgresql-$POSTGRESQL_VERSION.tar.bz2
  6 +test -d /tmp/postgresql-$POSTGRESQL_VERSION || tar zxvf postgresql-$POSTGRESQL_VERSION.tar.bz2
  7 +cd postgresql-$POSTGRESQL_VERSION && eval ./configure `pg_config --configure` && make
  8 +cd contrib/unaccent && make && make install
  9 +cd ..
  10 +cd contrib/pg_trgm && make && make install
  11 +cd ..
  12 +cd contrib/fuzzystrmatch && make && make install
... ...
plugins/pg_search/lib/pg_search/spec/associations_spec.rb 0 → 100644
... ... @@ -0,0 +1,316 @@
  1 +require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
  2 +
  3 +describe PgSearch do
  4 + context "joining to another table" do
  5 + if defined?(ActiveRecord::Relation)
  6 + context "with Arel support" do
  7 + context "without an :against" do
  8 + with_model :AssociatedModel do
  9 + table do |t|
  10 + t.string "title"
  11 + end
  12 + end
  13 +
  14 + with_model :ModelWithoutAgainst do
  15 + table do |t|
  16 + t.string "title"
  17 + t.belongs_to :another_model
  18 + end
  19 +
  20 + model do
  21 + include PgSearch
  22 + belongs_to :another_model, :class_name => 'AssociatedModel'
  23 +
  24 + pg_search_scope :with_another, :associated_against => {:another_model => :title}
  25 + end
  26 + end
  27 +
  28 + it "returns rows that match the query in the columns of the associated model only" do
  29 + associated = AssociatedModel.create!(:title => 'abcdef')
  30 + included = [
  31 + ModelWithoutAgainst.create!(:title => 'abcdef', :another_model => associated),
  32 + ModelWithoutAgainst.create!(:title => 'ghijkl', :another_model => associated)
  33 + ]
  34 + excluded = [
  35 + ModelWithoutAgainst.create!(:title => 'abcdef')
  36 + ]
  37 +
  38 + results = ModelWithoutAgainst.with_another('abcdef')
  39 + results.map(&:title).should =~ included.map(&:title)
  40 + results.should_not include(excluded)
  41 + end
  42 + end
  43 +
  44 + context "through a belongs_to association" do
  45 + with_model :AssociatedModel do
  46 + table do |t|
  47 + t.string 'title'
  48 + end
  49 + end
  50 +
  51 + with_model :ModelWithBelongsTo do
  52 + table do |t|
  53 + t.string 'title'
  54 + t.belongs_to 'another_model'
  55 + end
  56 +
  57 + model do
  58 + include PgSearch
  59 + belongs_to :another_model, :class_name => 'AssociatedModel'
  60 +
  61 + pg_search_scope :with_associated, :against => :title, :associated_against => {:another_model => :title}
  62 + end
  63 + end
  64 +
  65 + it "returns rows that match the query in either its own columns or the columns of the associated model" do
  66 + associated = AssociatedModel.create!(:title => 'abcdef')
  67 + included = [
  68 + ModelWithBelongsTo.create!(:title => 'ghijkl', :another_model => associated),
  69 + ModelWithBelongsTo.create!(:title => 'abcdef')
  70 + ]
  71 + excluded = ModelWithBelongsTo.create!(:title => 'mnopqr',
  72 + :another_model => AssociatedModel.create!(:title => 'stuvwx'))
  73 +
  74 + results = ModelWithBelongsTo.with_associated('abcdef')
  75 + results.map(&:title).should =~ included.map(&:title)
  76 + results.should_not include(excluded)
  77 + end
  78 + end
  79 +
  80 + context "through a has_many association" do
  81 + with_model :AssociatedModelWithHasMany do
  82 + table do |t|
  83 + t.string 'title'
  84 + t.belongs_to 'ModelWithHasMany'
  85 + end
  86 + end
  87 +
  88 + with_model :ModelWithHasMany do
  89 + table do |t|
  90 + t.string 'title'
  91 + end
  92 +
  93 + model do
  94 + include PgSearch
  95 + has_many :other_models, :class_name => 'AssociatedModelWithHasMany', :foreign_key => 'ModelWithHasMany_id'
  96 +
  97 + pg_search_scope :with_associated, :against => [:title], :associated_against => {:other_models => :title}
  98 + end
  99 + end
  100 +
  101 + it "returns rows that match the query in either its own columns or the columns of the associated model" do
  102 + included = [
  103 + ModelWithHasMany.create!(:title => 'abcdef', :other_models => [
  104 + AssociatedModelWithHasMany.create!(:title => 'foo'),
  105 + AssociatedModelWithHasMany.create!(:title => 'bar')
  106 + ]),
  107 + ModelWithHasMany.create!(:title => 'ghijkl', :other_models => [
  108 + AssociatedModelWithHasMany.create!(:title => 'foo bar'),
  109 + AssociatedModelWithHasMany.create!(:title => 'mnopqr')
  110 + ]),
  111 + ModelWithHasMany.create!(:title => 'foo bar')
  112 + ]
  113 + excluded = ModelWithHasMany.create!(:title => 'stuvwx', :other_models => [
  114 + AssociatedModelWithHasMany.create!(:title => 'abcdef')
  115 + ])
  116 +
  117 + results = ModelWithHasMany.with_associated('foo bar')
  118 + results.map(&:title).should =~ included.map(&:title)
  119 + results.should_not include(excluded)
  120 + end
  121 + end
  122 +
  123 + context "across multiple associations" do
  124 + context "on different tables" do
  125 + with_model :FirstAssociatedModel do
  126 + table do |t|
  127 + t.string 'title'
  128 + t.belongs_to 'model_with_many_associations'
  129 + end
  130 + model {}
  131 + end
  132 +
  133 + with_model :SecondAssociatedModel do
  134 + table do |t|
  135 + t.string 'title'
  136 + end
  137 + model {}
  138 + end
  139 +
  140 + with_model :ModelWithManyAssociations do
  141 + table do |t|
  142 + t.string 'title'
  143 + t.belongs_to 'model_of_second_type'
  144 + end
  145 +
  146 + model do
  147 + include PgSearch
  148 + has_many :models_of_first_type, :class_name => 'FirstAssociatedModel', :foreign_key => 'model_with_many_associations_id'
  149 + belongs_to :model_of_second_type, :class_name => 'SecondAssociatedModel'
  150 +
  151 + pg_search_scope :with_associated, :against => :title,
  152 + :associated_against => {:models_of_first_type => :title, :model_of_second_type => :title}
  153 + end
  154 + end
  155 +
  156 + it "returns rows that match the query in either its own columns or the columns of the associated model" do
  157 + matching_second = SecondAssociatedModel.create!(:title => "foo bar")
  158 + unmatching_second = SecondAssociatedModel.create!(:title => "uiop")
  159 +
  160 + included = [
  161 + ModelWithManyAssociations.create!(:title => 'abcdef', :models_of_first_type => [
  162 + FirstAssociatedModel.create!(:title => 'foo'),
  163 + FirstAssociatedModel.create!(:title => 'bar')
  164 + ]),
  165 + ModelWithManyAssociations.create!(:title => 'ghijkl', :models_of_first_type => [
  166 + FirstAssociatedModel.create!(:title => 'foo bar'),
  167 + FirstAssociatedModel.create!(:title => 'mnopqr')
  168 + ]),
  169 + ModelWithManyAssociations.create!(:title => 'foo bar'),
  170 + ModelWithManyAssociations.create!(:title => 'qwerty', :model_of_second_type => matching_second)
  171 + ]
  172 + excluded = [
  173 + ModelWithManyAssociations.create!(:title => 'stuvwx', :models_of_first_type => [
  174 + FirstAssociatedModel.create!(:title => 'abcdef')
  175 + ]),
  176 + ModelWithManyAssociations.create!(:title => 'qwerty', :model_of_second_type => unmatching_second)
  177 + ]
  178 +
  179 + results = ModelWithManyAssociations.with_associated('foo bar')
  180 + results.map(&:title).should =~ included.map(&:title)
  181 + excluded.each { |object| results.should_not include(object) }
  182 + end
  183 + end
  184 +
  185 + context "on the same table" do
  186 + with_model :DoublyAssociatedModel do
  187 + table do |t|
  188 + t.string 'title'
  189 + t.belongs_to 'model_with_double_association'
  190 + t.belongs_to 'model_with_double_association_again'
  191 + end
  192 + model {}
  193 + end
  194 +
  195 + with_model :model_with_double_association do
  196 + table do |t|
  197 + t.string 'title'
  198 + end
  199 +
  200 + model do
  201 + include PgSearch
  202 + has_many :things, :class_name => 'DoublyAssociatedModel', :foreign_key => 'model_with_double_association_id'
  203 + has_many :thingamabobs, :class_name => 'DoublyAssociatedModel', :foreign_key => 'model_with_double_association_again_id'
  204 +
  205 + pg_search_scope :with_associated, :against => :title,
  206 + :associated_against => {:things => :title, :thingamabobs => :title}
  207 + end
  208 + end
  209 +
  210 + it "returns rows that match the query in either its own columns or the columns of the associated model" do
  211 + included = [
  212 + ModelWithDoubleAssociation.create!(:title => 'abcdef', :things => [
  213 + DoublyAssociatedModel.create!(:title => 'foo'),
  214 + DoublyAssociatedModel.create!(:title => 'bar')
  215 + ]),
  216 + ModelWithDoubleAssociation.create!(:title => 'ghijkl', :things => [
  217 + DoublyAssociatedModel.create!(:title => 'foo bar'),
  218 + DoublyAssociatedModel.create!(:title => 'mnopqr')
  219 + ]),
  220 + ModelWithDoubleAssociation.create!(:title => 'foo bar'),
  221 + ModelWithDoubleAssociation.create!(:title => 'qwerty', :thingamabobs => [
  222 + DoublyAssociatedModel.create!(:title => "foo bar")
  223 + ])
  224 + ]
  225 + excluded = [
  226 + ModelWithDoubleAssociation.create!(:title => 'stuvwx', :things => [
  227 + DoublyAssociatedModel.create!(:title => 'abcdef')
  228 + ]),
  229 + ModelWithDoubleAssociation.create!(:title => 'qwerty', :thingamabobs => [
  230 + DoublyAssociatedModel.create!(:title => "uiop")
  231 + ])
  232 + ]
  233 +
  234 + results = ModelWithDoubleAssociation.with_associated('foo bar')
  235 + results.map(&:title).should =~ included.map(&:title)
  236 + excluded.each { |object| results.should_not include(object) }
  237 + end
  238 + end
  239 + end
  240 +
  241 + context "against multiple attributes on one association" do
  242 + with_model :AssociatedModel do
  243 + table do |t|
  244 + t.string 'title'
  245 + t.text 'author'
  246 + end
  247 + end
  248 +
  249 + with_model :ModelWithAssociation do
  250 + table do |t|
  251 + t.belongs_to 'another_model'
  252 + end
  253 +
  254 + model do
  255 + include PgSearch
  256 + belongs_to :another_model, :class_name => 'AssociatedModel'
  257 +
  258 + pg_search_scope :with_associated, :associated_against => {:another_model => [:title, :author]}
  259 + end
  260 + end
  261 +
  262 + it "should only do one join" do
  263 + included = [
  264 + ModelWithAssociation.create!(
  265 + :another_model => AssociatedModel.create!(
  266 + :title => "foo",
  267 + :author => "bar"
  268 + )
  269 + ),
  270 + ModelWithAssociation.create!(
  271 + :another_model => AssociatedModel.create!(
  272 + :title => "foo bar",
  273 + :author => "baz"
  274 + )
  275 + )
  276 + ]
  277 + excluded = [
  278 + ModelWithAssociation.create!(
  279 + :another_model => AssociatedModel.create!(
  280 + :title => "foo",
  281 + :author => "baz"
  282 + )
  283 + )
  284 + ]
  285 +
  286 + results = ModelWithAssociation.with_associated('foo bar')
  287 +
  288 + results.to_sql.scan("INNER JOIN").length.should == 1
  289 + included.each { |object| results.should include(object) }
  290 + excluded.each { |object| results.should_not include(object) }
  291 + end
  292 +
  293 + end
  294 + end
  295 + else
  296 + context "without Arel support" do
  297 + with_model :Model do
  298 + table do |t|
  299 + t.string 'title'
  300 + end
  301 +
  302 + model do
  303 + include PgSearch
  304 + pg_search_scope :with_joins, :against => :title, :joins => :another_model
  305 + end
  306 + end
  307 +
  308 + it "should raise an error" do
  309 + lambda {
  310 + Model.with_joins('foo')
  311 + }.should raise_error(ArgumentError, /joins/)
  312 + end
  313 + end
  314 + end
  315 + end
  316 +end
... ...
plugins/pg_search/lib/pg_search/spec/pg_search_spec.rb 0 → 100644
... ... @@ -0,0 +1,648 @@
  1 +require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
  2 +
  3 +describe "an ActiveRecord model which includes PgSearch" do
  4 +
  5 + with_model :ModelWithPgSearch do
  6 + table do |t|
  7 + t.string 'title'
  8 + t.text 'content'
  9 + t.integer 'importance'
  10 + end
  11 +
  12 + model do
  13 + include PgSearch
  14 + end
  15 + end
  16 +
  17 + describe ".pg_search_scope" do
  18 + it "builds a scope" do
  19 + ModelWithPgSearch.class_eval do
  20 + pg_search_scope "matching_query", :against => []
  21 + end
  22 +
  23 + lambda {
  24 + ModelWithPgSearch.scoped({}).matching_query("foo").scoped({})
  25 + }.should_not raise_error
  26 + end
  27 +
  28 + context "when passed a lambda" do
  29 + it "builds a dynamic scope" do
  30 + ModelWithPgSearch.class_eval do
  31 + pg_search_scope :search_title_or_content, lambda { |query, pick_content|
  32 + {
  33 + :query => query.gsub("-remove-", ""),
  34 + :against => pick_content ? :content : :title
  35 + }
  36 + }
  37 + end
  38 +
  39 + included = ModelWithPgSearch.create!(:title => 'foo', :content => 'bar')
  40 + excluded = ModelWithPgSearch.create!(:title => 'bar', :content => 'foo')
  41 +
  42 + ModelWithPgSearch.search_title_or_content('fo-remove-o', false).should == [included]
  43 + ModelWithPgSearch.search_title_or_content('b-remove-ar', true).should == [included]
  44 + end
  45 + end
  46 +
  47 + context "when an unknown option is passed in" do
  48 + it "raises an exception when invoked" do
  49 + lambda {
  50 + ModelWithPgSearch.class_eval do
  51 + pg_search_scope :with_unknown_option, :against => :content, :foo => :bar
  52 + end
  53 + ModelWithPgSearch.with_unknown_option("foo")
  54 + }.should raise_error(ArgumentError, /foo/)
  55 + end
  56 +
  57 + context "dynamically" do
  58 + it "raises an exception when invoked" do
  59 + lambda {
  60 + ModelWithPgSearch.class_eval do
  61 + pg_search_scope :with_unknown_option, lambda { |*| {:against => :content, :foo => :bar} }
  62 + end
  63 + ModelWithPgSearch.with_unknown_option("foo")
  64 + }.should raise_error(ArgumentError, /foo/)
  65 + end
  66 + end
  67 + end
  68 +
  69 + context "when an unknown :using is passed" do
  70 + it "raises an exception when invoked" do
  71 + lambda {
  72 + ModelWithPgSearch.class_eval do
  73 + pg_search_scope :with_unknown_using, :against => :content, :using => :foo
  74 + end
  75 + ModelWithPgSearch.with_unknown_using("foo")
  76 + }.should raise_error(ArgumentError, /foo/)
  77 + end
  78 +
  79 + context "dynamically" do
  80 + it "raises an exception when invoked" do
  81 + lambda {
  82 + ModelWithPgSearch.class_eval do
  83 + pg_search_scope :with_unknown_using, lambda { |*| {:against => :content, :using => :foo} }
  84 + end
  85 + ModelWithPgSearch.with_unknown_using("foo")
  86 + }.should raise_error(ArgumentError, /foo/)
  87 + end
  88 + end
  89 + end
  90 +
  91 + context "when an unknown :ignoring is passed" do
  92 + it "raises an exception when invoked" do
  93 + lambda {
  94 + ModelWithPgSearch.class_eval do
  95 + pg_search_scope :with_unknown_ignoring, :against => :content, :ignoring => :foo
  96 + end
  97 + ModelWithPgSearch.with_unknown_ignoring("foo")
  98 + }.should raise_error(ArgumentError, /ignoring.*foo/)
  99 + end
  100 +
  101 + context "dynamically" do
  102 + it "raises an exception when invoked" do
  103 + lambda {
  104 + ModelWithPgSearch.class_eval do
  105 + pg_search_scope :with_unknown_ignoring, lambda { |*| {:against => :content, :ignoring => :foo} }
  106 + end
  107 + ModelWithPgSearch.with_unknown_ignoring("foo")
  108 + }.should raise_error(ArgumentError, /ignoring.*foo/)
  109 + end
  110 + end
  111 +
  112 + context "when :against is not passed in" do
  113 + it "raises an exception when invoked" do
  114 + lambda {
  115 + ModelWithPgSearch.class_eval do
  116 + pg_search_scope :with_unknown_ignoring, {}
  117 + end
  118 + ModelWithPgSearch.with_unknown_ignoring("foo")
  119 + }.should raise_error(ArgumentError, /against/)
  120 + end
  121 + context "dynamically" do
  122 + it "raises an exception when invoked" do
  123 + lambda {
  124 + ModelWithPgSearch.class_eval do
  125 + pg_search_scope :with_unknown_ignoring, lambda { |*| {} }
  126 + end
  127 + ModelWithPgSearch.with_unknown_ignoring("foo")
  128 + }.should raise_error(ArgumentError, /against/)
  129 + end
  130 + end
  131 + end
  132 + end
  133 + end
  134 +
  135 + describe "a search scope" do
  136 + context "against a single column" do
  137 + before do
  138 + ModelWithPgSearch.class_eval do
  139 + pg_search_scope :search_content, :against => :content
  140 + end
  141 + end
  142 +
  143 + it "returns an empty array when a blank query is passed in" do
  144 + ModelWithPgSearch.create!(:content => 'foo')
  145 +
  146 + results = ModelWithPgSearch.search_content('')
  147 + results.should == []
  148 + end
  149 +
  150 + it "returns rows where the column contains the term in the query" do
  151 + included = ModelWithPgSearch.create!(:content => 'foo')
  152 + excluded = ModelWithPgSearch.create!(:content => 'bar')
  153 +
  154 + results = ModelWithPgSearch.search_content('foo')
  155 + results.should include(included)
  156 + results.should_not include(excluded)
  157 + end
  158 +
  159 + it "returns rows where the column contains all the terms in the query in any order" do
  160 + included = [ModelWithPgSearch.create!(:content => 'foo bar'),
  161 + ModelWithPgSearch.create!(:content => 'bar foo')]
  162 + excluded = ModelWithPgSearch.create!(:content => 'foo')
  163 +
  164 + results = ModelWithPgSearch.search_content('foo bar')
  165 + results.should =~ included
  166 + results.should_not include(excluded)
  167 + end
  168 +
  169 + it "returns rows that match the query but not its case" do
  170 + # \303\241 is a with acute accent
  171 + # \303\251 is e with acute accent
  172 +
  173 + included = [ModelWithPgSearch.create!(:content => "foo"),
  174 + ModelWithPgSearch.create!(:content => "FOO")]
  175 +
  176 + results = ModelWithPgSearch.search_content("Foo")
  177 + results.should =~ included
  178 + end
  179 +
  180 + it "returns rows that match the query only if their accents match" do
  181 + # \303\241 is a with acute accent
  182 + # \303\251 is e with acute accent
  183 +
  184 + included = ModelWithPgSearch.create!(:content => "abcd\303\251f")
  185 + excluded = ModelWithPgSearch.create!(:content => "\303\241bcdef")
  186 +
  187 + results = ModelWithPgSearch.search_content("abcd\303\251f")
  188 + results.should == [included]
  189 + results.should_not include(excluded)
  190 + end
  191 +
  192 + it "returns rows that match the query but not rows that are prefixed by the query" do
  193 + included = ModelWithPgSearch.create!(:content => 'pre')
  194 + excluded = ModelWithPgSearch.create!(:content => 'prefix')
  195 +
  196 + results = ModelWithPgSearch.search_content("pre")
  197 + results.should == [included]
  198 + results.should_not include(excluded)
  199 + end
  200 +
  201 + it "returns rows that match the query exactly and not those that match the query when stemmed by the default english dictionary" do
  202 + included = ModelWithPgSearch.create!(:content => "jumped")
  203 + excluded = [ModelWithPgSearch.create!(:content => "jump"),
  204 + ModelWithPgSearch.create!(:content => "jumping")]
  205 +
  206 + results = ModelWithPgSearch.search_content("jumped")
  207 + results.should == [included]
  208 + end
  209 +
  210 + it "returns rows that match sorted by rank" do
  211 + loser = ModelWithPgSearch.create!(:content => 'foo')
  212 + winner = ModelWithPgSearch.create!(:content => 'foo foo')
  213 +
  214 + results = ModelWithPgSearch.search_content("foo")
  215 + results[0].rank.should > results[1].rank
  216 + results.should == [winner, loser]
  217 + end
  218 +
  219 + it "returns results that match sorted by primary key for records that rank the same" do
  220 + sorted_results = [ModelWithPgSearch.create!(:content => 'foo'),
  221 + ModelWithPgSearch.create!(:content => 'foo')].sort_by(&:id)
  222 +
  223 + results = ModelWithPgSearch.search_content("foo")
  224 + results.should == sorted_results
  225 + end
  226 +
  227 + it "returns results that match a query with multiple space-separated search terms" do
  228 + included = [
  229 + ModelWithPgSearch.create!(:content => 'foo bar'),
  230 + ModelWithPgSearch.create!(:content => 'bar foo'),
  231 + ModelWithPgSearch.create!(:content => 'bar foo baz'),
  232 + ]
  233 + excluded = [
  234 + ModelWithPgSearch.create!(:content => 'foo'),
  235 + ModelWithPgSearch.create!(:content => 'foo baz')
  236 + ]
  237 +
  238 + results = ModelWithPgSearch.search_content('foo bar')
  239 + results.should =~ included
  240 + results.should_not include(excluded)
  241 + end
  242 +
  243 + it "returns rows that match a query with characters that are invalid in a tsquery expression" do
  244 + included = ModelWithPgSearch.create!(:content => "(:Foo.) Bar?, \\")
  245 +
  246 + results = ModelWithPgSearch.search_content("foo :bar .,?() \\")
  247 + results.should == [included]
  248 + end
  249 +
  250 + it "accepts non-string queries and calls #to_s on them" do
  251 + foo = ModelWithPgSearch.create!(:content => "foo")
  252 + not_a_string = stub(:to_s => "foo")
  253 + ModelWithPgSearch.search_content(not_a_string).should == [foo]
  254 + end
  255 + end
  256 +
  257 + context "against multiple columns" do
  258 + before do
  259 + ModelWithPgSearch.class_eval do
  260 + pg_search_scope :search_title_and_content, :against => [:title, :content]
  261 + end
  262 + end
  263 +
  264 + it "returns rows whose columns contain all of the terms in the query across columns" do
  265 + included = [
  266 + ModelWithPgSearch.create!(:title => 'foo', :content => 'bar'),
  267 + ModelWithPgSearch.create!(:title => 'bar', :content => 'foo')
  268 + ]
  269 + excluded = [
  270 + ModelWithPgSearch.create!(:title => 'foo', :content => 'foo'),
  271 + ModelWithPgSearch.create!(:title => 'bar', :content => 'bar')
  272 + ]
  273 +
  274 + results = ModelWithPgSearch.search_title_and_content('foo bar')
  275 +
  276 + results.should =~ included
  277 + excluded.each do |result|
  278 + results.should_not include(result)
  279 + end
  280 + end
  281 +
  282 + it "returns rows where at one column contains all of the terms in the query and another does not" do
  283 + in_title = ModelWithPgSearch.create!(:title => 'foo', :content => 'bar')
  284 + in_content = ModelWithPgSearch.create!(:title => 'bar', :content => 'foo')
  285 +
  286 + results = ModelWithPgSearch.search_title_and_content('foo')
  287 + results.should =~ [in_title, in_content]
  288 + end
  289 +
  290 + # Searching with a NULL column will prevent any matches unless we coalesce it.
  291 + it "returns rows where at one column contains all of the terms in the query and another is NULL" do
  292 + included = ModelWithPgSearch.create!(:title => 'foo', :content => nil)
  293 + results = ModelWithPgSearch.search_title_and_content('foo')
  294 + results.should == [included]
  295 + end
  296 + end
  297 +
  298 + context "using trigram" do
  299 + before do
  300 + ModelWithPgSearch.class_eval do
  301 + pg_search_scope :with_trigrams, :against => [:title, :content], :using => :trigram
  302 + end
  303 + end
  304 +
  305 + it "returns rows where one searchable column and the query share enough trigrams" do
  306 + included = ModelWithPgSearch.create!(:title => 'abcdefghijkl', :content => nil)
  307 + results = ModelWithPgSearch.with_trigrams('cdefhijkl')
  308 + results.should == [included]
  309 + end
  310 +
  311 + it "returns rows where multiple searchable columns and the query share enough trigrams" do
  312 + included = ModelWithPgSearch.create!(:title => 'abcdef', :content => 'ghijkl')
  313 + results = ModelWithPgSearch.with_trigrams('cdefhijkl')
  314 + results.should == [included]
  315 + end
  316 + end
  317 +
  318 + context "using tsearch" do
  319 + context "with :prefix => true" do
  320 + before do
  321 + ModelWithPgSearch.class_eval do
  322 + pg_search_scope :search_title_with_prefixes,
  323 + :against => :title,
  324 + :using => {
  325 + :tsearch => {:prefix => true}
  326 + }
  327 + end
  328 + end
  329 +
  330 + it "returns rows that match the query and that are prefixed by the query" do
  331 + included = ModelWithPgSearch.create!(:title => 'prefix')
  332 + excluded = ModelWithPgSearch.create!(:title => 'postfix')
  333 +
  334 + results = ModelWithPgSearch.search_title_with_prefixes("pre")
  335 + results.should == [included]
  336 + results.should_not include(excluded)
  337 + end
  338 +
  339 + it "returns rows that match the query when the query has a hyphen" do
  340 + included = [
  341 + ModelWithPgSearch.create!(:title => 'foo bar'),
  342 + ModelWithPgSearch.create!(:title => 'foo-bar')
  343 + ]
  344 + excluded = ModelWithPgSearch.create!(:title => 'baz quux')
  345 +
  346 + results = ModelWithPgSearch.search_title_with_prefixes("foo-bar")
  347 + results.should =~ included
  348 + results.should_not include(excluded)
  349 + end
  350 + end
  351 +
  352 + context "with the english dictionary" do
  353 + before do
  354 + ModelWithPgSearch.class_eval do
  355 + pg_search_scope :search_content_with_english,
  356 + :against => :content,
  357 + :using => {
  358 + :tsearch => {:dictionary => :english}
  359 + }
  360 + end
  361 + end
  362 +
  363 + it "returns rows that match the query when stemmed by the english dictionary" do
  364 + included = [ModelWithPgSearch.create!(:content => "jump"),
  365 + ModelWithPgSearch.create!(:content => "jumped"),
  366 + ModelWithPgSearch.create!(:content => "jumping")]
  367 +
  368 + results = ModelWithPgSearch.search_content_with_english("jump")
  369 + results.should =~ included
  370 + end
  371 + end
  372 +
  373 + context "against columns ranked with arrays" do
  374 + before do
  375 + ModelWithPgSearch.class_eval do
  376 + pg_search_scope :search_weighted_by_array_of_arrays, :against => [[:content, 'B'], [:title, 'A']]
  377 + end
  378 + end
  379 +
  380 + it "returns results sorted by weighted rank" do
  381 + loser = ModelWithPgSearch.create!(:title => 'bar', :content => 'foo')
  382 + winner = ModelWithPgSearch.create!(:title => 'foo', :content => 'bar')
  383 +
  384 + results = ModelWithPgSearch.search_weighted_by_array_of_arrays('foo')
  385 + results[0].rank.should > results[1].rank
  386 + results.should == [winner, loser]
  387 + end
  388 + end
  389 +
  390 + context "against columns ranked with a hash" do
  391 + before do
  392 + ModelWithPgSearch.class_eval do
  393 + pg_search_scope :search_weighted_by_hash, :against => {:content => 'B', :title => 'A'}
  394 + end
  395 + end
  396 +
  397 + it "returns results sorted by weighted rank" do
  398 + loser = ModelWithPgSearch.create!(:title => 'bar', :content => 'foo')
  399 + winner = ModelWithPgSearch.create!(:title => 'foo', :content => 'bar')
  400 +
  401 + results = ModelWithPgSearch.search_weighted_by_hash('foo')
  402 + results[0].rank.should > results[1].rank
  403 + results.should == [winner, loser]
  404 + end
  405 + end
  406 +
  407 + context "against columns of which only some are ranked" do
  408 + before do
  409 + ModelWithPgSearch.class_eval do
  410 + pg_search_scope :search_weighted, :against => [:content, [:title, 'A']]
  411 + end
  412 + end
  413 +
  414 + it "returns results sorted by weighted rank using an implied low rank for unranked columns" do
  415 + loser = ModelWithPgSearch.create!(:title => 'bar', :content => 'foo')
  416 + winner = ModelWithPgSearch.create!(:title => 'foo', :content => 'bar')
  417 +
  418 + results = ModelWithPgSearch.search_weighted('foo')
  419 + results[0].rank.should > results[1].rank
  420 + results.should == [winner, loser]
  421 + end
  422 + end
  423 + end
  424 +
  425 + context "using dmetaphone" do
  426 + before do
  427 + ModelWithPgSearch.class_eval do
  428 + pg_search_scope :with_dmetaphones, :against => [:title, :content], :using => :dmetaphone
  429 + end
  430 + end
  431 +
  432 + it "returns rows where one searchable column and the query share enough dmetaphones" do
  433 + included = ModelWithPgSearch.create!(:title => 'Geoff', :content => nil)
  434 + excluded = ModelWithPgSearch.create!(:title => 'Bob', :content => nil)
  435 + results = ModelWithPgSearch.with_dmetaphones('Jeff')
  436 + results.should == [included]
  437 + end
  438 +
  439 + it "returns rows where multiple searchable columns and the query share enough dmetaphones" do
  440 + included = ModelWithPgSearch.create!(:title => 'Geoff', :content => 'George')
  441 + excluded = ModelWithPgSearch.create!(:title => 'Bob', :content => 'Jones')
  442 + results = ModelWithPgSearch.with_dmetaphones('Jeff Jorge')
  443 + results.should == [included]
  444 + end
  445 +
  446 + it "returns rows that match dmetaphones that are English stopwords" do
  447 + included = ModelWithPgSearch.create!(:title => 'White', :content => nil)
  448 + excluded = ModelWithPgSearch.create!(:title => 'Black', :content => nil)
  449 + results = ModelWithPgSearch.with_dmetaphones('Wight')
  450 + results.should == [included]
  451 + end
  452 +
  453 + it "can handle terms that do not have a dmetaphone equivalent" do
  454 + term_with_blank_metaphone = "w"
  455 +
  456 + included = ModelWithPgSearch.create!(:title => 'White', :content => nil)
  457 + excluded = ModelWithPgSearch.create!(:title => 'Black', :content => nil)
  458 +
  459 + results = ModelWithPgSearch.with_dmetaphones('Wight W')
  460 + results.should == [included]
  461 + end
  462 + end
  463 +
  464 + context "using multiple features" do
  465 + before do
  466 + ModelWithPgSearch.class_eval do
  467 + pg_search_scope :with_tsearch,
  468 + :against => :title,
  469 + :using => [
  470 + [:tsearch, {:prefix => true}]
  471 + ]
  472 +
  473 + pg_search_scope :with_trigram, :against => :title, :using => :trigram
  474 +
  475 + pg_search_scope :with_tsearch_and_trigram_using_array,
  476 + :against => :title,
  477 + :using => [
  478 + [:tsearch, {:prefix => true}],
  479 + :trigram
  480 + ]
  481 +
  482 + end
  483 + end
  484 +
  485 + it "returns rows that match using any of the features" do
  486 + record = ModelWithPgSearch.create!(:title => "tiling is grouty")
  487 +
  488 + # matches trigram only
  489 + trigram_query = "ling is grouty"
  490 + ModelWithPgSearch.with_trigram(trigram_query).should include(record)
  491 + ModelWithPgSearch.with_tsearch(trigram_query).should_not include(record)
  492 + ModelWithPgSearch.with_tsearch_and_trigram_using_array(trigram_query).should == [record]
  493 +
  494 + # matches tsearch only
  495 + tsearch_query = "til"
  496 + ModelWithPgSearch.with_tsearch(tsearch_query).should include(record)
  497 + ModelWithPgSearch.with_trigram(tsearch_query).should_not include(record)
  498 + ModelWithPgSearch.with_tsearch_and_trigram_using_array(tsearch_query).should == [record]
  499 + end
  500 +
  501 + context "with feature-specific configuration" do
  502 + before do
  503 + @tsearch_config = tsearch_config = {:dictionary => 'english'}
  504 + @trigram_config = trigram_config = {:foo => 'bar'}
  505 +
  506 + ModelWithPgSearch.class_eval do
  507 + pg_search_scope :with_tsearch_and_trigram_using_hash,
  508 + :against => :title,
  509 + :using => {
  510 + :tsearch => tsearch_config,
  511 + :trigram => trigram_config
  512 + }
  513 + end
  514 + end
  515 +
  516 + it "should pass the custom configuration down to the specified feature" do
  517 + stub_feature = stub(:conditions => "1 = 1", :rank => "1.0")
  518 + PgSearch::Features::TSearch.should_receive(:new).with(anything, @tsearch_config, anything, anything, anything).at_least(:once).and_return(stub_feature)
  519 + PgSearch::Features::Trigram.should_receive(:new).with(anything, @trigram_config, anything, anything, anything).at_least(:once).and_return(stub_feature)
  520 +
  521 + ModelWithPgSearch.with_tsearch_and_trigram_using_hash("foo")
  522 + end
  523 + end
  524 + end
  525 +
  526 + context "using a tsvector column" do
  527 + with_model :ModelWithPgSearchUsingTsVectorColumn do
  528 + table do |t|
  529 + t.text 'content'
  530 + t.column 'content_tsvector', :tsvector
  531 + end
  532 +
  533 + model { include PgSearch }
  534 + end
  535 +
  536 + let!(:expected) { ModelWithPgSearchUsingTsVectorColumn.create!(:content => 'tiling is grouty') }
  537 + let!(:unexpected) { ModelWithPgSearchUsingTsVectorColumn.create!(:content => 'longcat is looooooooong') }
  538 +
  539 + before do
  540 + ActiveRecord::Base.connection.execute <<-SQL
  541 + UPDATE #{ModelWithPgSearchUsingTsVectorColumn.table_name}
  542 + SET content_tsvector = to_tsvector('english'::regconfig, "#{ModelWithPgSearchUsingTsVectorColumn.table_name}"."content")
  543 + SQL
  544 +
  545 + ModelWithPgSearchUsingTsVectorColumn.class_eval do
  546 + pg_search_scope :search_by_content_with_tsvector,
  547 + :against => :content,
  548 + :using => {
  549 + :tsearch => {
  550 + :tsvector_column => 'content_tsvector',
  551 + :dictionary => 'english'
  552 + }
  553 + }
  554 + end
  555 + end
  556 +
  557 + if defined?(ActiveRecord::Relation)
  558 + it "should not use to_tsvector in the query" do
  559 + ModelWithPgSearchUsingTsVectorColumn.search_by_content_with_tsvector("tiles").to_sql.should_not =~ /to_tsvector/
  560 + end
  561 + end
  562 +
  563 + it "should find the expected result" do
  564 + ModelWithPgSearchUsingTsVectorColumn.search_by_content_with_tsvector("tiles").map(&:id).should == [expected.id]
  565 + end
  566 + end
  567 +
  568 + context "ignoring accents" do
  569 + before do
  570 + ModelWithPgSearch.class_eval do
  571 + pg_search_scope :search_title_without_accents, :against => :title, :ignoring => :accents
  572 + end
  573 + end
  574 +
  575 + it "returns rows that match the query but not its accents" do
  576 + # \303\241 is a with acute accent
  577 + # \303\251 is e with acute accent
  578 +
  579 + included = ModelWithPgSearch.create!(:title => "\303\241bcdef")
  580 +
  581 + results = ModelWithPgSearch.search_title_without_accents("abcd\303\251f")
  582 + results.should == [included]
  583 + end
  584 + end
  585 +
  586 + context "when passed a :ranked_by expression" do
  587 + before do
  588 + ModelWithPgSearch.class_eval do
  589 + pg_search_scope :search_content_with_default_rank,
  590 + :against => :content
  591 + pg_search_scope :search_content_with_importance_as_rank,
  592 + :against => :content,
  593 + :ranked_by => "importance"
  594 + pg_search_scope :search_content_with_importance_as_rank_multiplier,
  595 + :against => :content,
  596 + :ranked_by => ":tsearch * importance"
  597 + end
  598 + end
  599 +
  600 + it "should return records with a rank attribute equal to the :ranked_by expression" do
  601 + ModelWithPgSearch.create!(:content => 'foo', :importance => 10)
  602 + results = ModelWithPgSearch.search_content_with_importance_as_rank("foo")
  603 + results.first.rank.should == 10
  604 + end
  605 +
  606 + it "should substitute :tsearch with the tsearch rank expression in the :ranked_by expression" do
  607 + ModelWithPgSearch.create!(:content => 'foo', :importance => 10)
  608 +
  609 + tsearch_rank = ModelWithPgSearch.search_content_with_default_rank("foo").first.rank
  610 + multiplied_rank = ModelWithPgSearch.search_content_with_importance_as_rank_multiplier("foo").first.rank
  611 +
  612 + multiplied_rank.should be_within(0.001).of(tsearch_rank * 10)
  613 + end
  614 +
  615 + it "should return results in descending order of the value of the rank expression" do
  616 + records = [
  617 + ModelWithPgSearch.create!(:content => 'foo', :importance => 1),
  618 + ModelWithPgSearch.create!(:content => 'foo', :importance => 3),
  619 + ModelWithPgSearch.create!(:content => 'foo', :importance => 2)
  620 + ]
  621 +
  622 + results = ModelWithPgSearch.search_content_with_importance_as_rank("foo")
  623 + results.should == records.sort_by(&:importance).reverse
  624 + end
  625 +
  626 + %w[tsearch trigram dmetaphone].each do |feature|
  627 +
  628 + context "using the #{feature} ranking algorithm" do
  629 + before do
  630 + @scope_name = scope_name = :"search_content_ranked_by_#{feature}"
  631 + ModelWithPgSearch.class_eval do
  632 + pg_search_scope scope_name,
  633 + :against => :content,
  634 + :ranked_by => ":#{feature}"
  635 + end
  636 + end
  637 +
  638 + it "should return results with a rank" do
  639 + ModelWithPgSearch.create!(:content => 'foo')
  640 +
  641 + results = ModelWithPgSearch.send(@scope_name, 'foo')
  642 + results.first.rank.should_not be_nil
  643 + end
  644 + end
  645 + end
  646 + end
  647 + end
  648 +end
... ...
plugins/pg_search/lib/pg_search/spec/spec_helper.rb 0 → 100644
... ... @@ -0,0 +1,92 @@
  1 +require "bundler/setup"
  2 +require "pg_search"
  3 +
  4 +begin
  5 + ActiveRecord::Base.establish_connection(:adapter => 'postgresql',
  6 + :database => 'pg_search_test',
  7 + :min_messages => 'warning')
  8 + connection = ActiveRecord::Base.connection
  9 + connection.execute("SELECT 1")
  10 +rescue PGError => e
  11 + puts "-" * 80
  12 + puts "Unable to connect to database. Please run:"
  13 + puts
  14 + puts " createdb pg_search_test"
  15 + puts "-" * 80
  16 + raise e
  17 +end
  18 +
  19 +def install_contrib_module_if_missing(name, query, expected_result)
  20 + connection = ActiveRecord::Base.connection
  21 + result = connection.select_value(query)
  22 + raise "Unexpected output for #{query}: #{result.inspect}" unless result.downcase == expected_result.downcase
  23 +rescue => e
  24 + begin
  25 + share_path = `pg_config --sharedir`.strip
  26 + ActiveRecord::Base.connection.execute File.read(File.join(share_path, 'contrib', "#{name}.sql"))
  27 + puts $!.message
  28 + rescue
  29 + puts "-" * 80
  30 + puts "Please install the #{name} contrib module"
  31 + puts "-" * 80
  32 + raise e
  33 + end
  34 +end
  35 +
  36 +install_contrib_module_if_missing("pg_trgm", "SELECT 'abcdef' % 'cdef'", "t")
  37 +install_contrib_module_if_missing("unaccent", "SELECT unaccent('foo')", "foo")
  38 +install_contrib_module_if_missing("fuzzystrmatch", "SELECT dmetaphone('foo')", "f")
  39 +
  40 +ActiveRecord::Base.connection.execute(File.read(File.join(File.dirname(__FILE__), '..', 'sql', 'dmetaphone.sql')))
  41 +
  42 +require "with_model"
  43 +
  44 +RSpec.configure do |config|
  45 + config.extend WithModel
  46 +end
  47 +
  48 +if defined?(ActiveRecord::Relation)
  49 + RSpec::Matchers::OperatorMatcher.register(ActiveRecord::Relation, '=~', RSpec::Matchers::MatchArray)
  50 +end
  51 +
  52 +require 'irb'
  53 +
  54 +class IRB::Irb
  55 + alias initialize_orig initialize
  56 + def initialize(workspace = nil, *args)
  57 + default = IRB.conf[:DEFAULT_OBJECT]
  58 + workspace ||= IRB::WorkSpace.new default if default
  59 + initialize_orig(workspace, *args)
  60 + end
  61 +end
  62 +
  63 +# Drop into an IRB session for whatever object you pass in:
  64 +#
  65 +# class Dude
  66 +# def abides
  67 +# true
  68 +# end
  69 +# end
  70 +#
  71 +# console_for(Dude.new)
  72 +#
  73 +# Then type "quit" or "exit" to get out. In a step definition, it should look like:
  74 +#
  75 +# When /^I console/ do
  76 +# console_for(self)
  77 +# end
  78 +#
  79 +# Also, I definitely stole this hack from some mailing list post somewhere. I wish I
  80 +# could remember who did it, but I can't. Sorry!
  81 +def console_for(target)
  82 + puts "== ENTERING CONSOLE MODE. ==\nType 'exit' to move on.\nContext: #{target.inspect}"
  83 +
  84 + begin
  85 + oldargs = ARGV.dup
  86 + ARGV.clear
  87 + IRB.conf[:DEFAULT_OBJECT] = target
  88 + IRB.start
  89 + ensure
  90 + ARGV.replace(oldargs)
  91 + end
  92 +end
... ...
plugins/pg_search/lib/pg_search/sql/dmetaphone.sql 0 → 100644
... ... @@ -0,0 +1,4 @@
  1 +CREATE OR REPLACE FUNCTION pg_search_dmetaphone(text) RETURNS text LANGUAGE SQL IMMUTABLE STRICT AS $function$
  2 + SELECT array_to_string(ARRAY(SELECT dmetaphone(unnest(regexp_split_to_array($1, E'\\\\s+')))), ' ')
  3 +$function$;
  4 +
... ...
plugins/pg_search/lib/pg_search/sql/uninstall_dmetaphone.sql 0 → 100644
... ... @@ -0,0 +1 @@
  1 +DROP FUNCTION pg_search_dmetaphone(text);
... ...
plugins/pg_search/lib/pg_search_plugin.rb 0 → 100644
... ... @@ -0,0 +1,31 @@
  1 +lib_path = File.join(File.dirname(__FILE__), 'pg_search', 'lib')
  2 +ActiveSupport::Dependencies.load_paths << lib_path
  3 +$: << lib_path
  4 +require 'pg_search'
  5 +
  6 +class PgSearchPlugin < Noosfero::Plugin
  7 +
  8 + def self.plugin_name
  9 + "Postgres Full-Text Search"
  10 + end
  11 +
  12 + def self.plugin_description
  13 + _("Search engine that uses Postgres Full-Text Search.")
  14 + end
  15 +
  16 + def find_by_contents(asset, scope, query, paginate_options={}, options={})
  17 + scope.pg_search_plugin_search(query)
  18 + end
  19 +
  20 +end
  21 +
  22 +searchables = %w[ article comment qualifier national_region certifier profile license scrap category ]
  23 +searchables.each { |searchable| require_dependency searchable }
  24 +klasses = searchables.map {|searchable| searchable.camelize.constantize }
  25 +
  26 +klasses.each do |klass|
  27 + klass.class_eval do
  28 + include PgSearch
  29 + pg_search_scope :pg_search_plugin_search, :against => klass::SEARCHABLE_FIELDS
  30 + end
  31 +end
... ...
plugins/pg_search/test/integration/performance_test.rb 0 → 100644
... ... @@ -0,0 +1,32 @@
  1 +require 'test_helper'
  2 +require 'benchmark'
  3 +
  4 +class PerformanceTest < ActionController::IntegrationTest
  5 +
  6 + searchables = %w[ article comment qualifier national_region certifier profile license scrap category ]
  7 + quantities = [10, 100, 1000]
  8 +
  9 + searchables.each do |searchable|
  10 + self.send(:define_method, "test_#{searchable}_performance") do
  11 + klass = searchable.camelize.constantize
  12 + asset = searchable.pluralize.to_sym
  13 + quantities.each do |quantity|
  14 + create(klass, quantity)
  15 + get 'index'
  16 + like = Benchmark.measure { 10.times { @controller.send(:find_by_contents, asset, klass, searchable) } }
  17 + puts "Like for #{quantity}: #{like}"
  18 + Environment.default.enable_plugin(PgSearchPlugin)
  19 + get 'index'
  20 + like = Benchmark.measure { 10.times { @controller.send(:find_by_contents, asset, klass, searchable) } }
  21 + puts "Pg for #{quantity}: #{pg}"
  22 + end
  23 + end
  24 + end
  25 +
  26 + private
  27 +
  28 + def create(klass, quantity)
  29 + klass.destroy_all
  30 + quantity.times.each {fast_create(klass)}
  31 + end
  32 +end
... ...