HOWTO: Adding search to your rails application

daibatzu · 2006-09-08 00:24:55

Sticky:: Please read the post below. I think you will find the next method even simpler.

This is a short guide on adding search to your Rails models.
For this tutorial, I'm using a model called Product and product has the following attributes: name, description, price. If you're new to rails, what this means is that you have a database table called products, and this table has the columns name, description and price (including id of course).

Now, the first thing to do is to install the ferret gem. Without this, this tutorial is moot. Open a command prompt and type:

Code : - fold - unfold

gem install ferret

Next, you will need to download the simple_search plugin for rails. The svn repositry is located at:

Code : - fold - unfold

http://julik.textdriven.com/svn/tools/rails_plugins/simple_search/

So you can get it through Subversion using the command

Code : - fold - unfold

svn co http://julik.textdriven.com/svn/tools/rails_plugins/simple_search

If you don't have Subversion you can download it from http://subversion.tigris.org/project_packages.html. After you download simple_search, place it in the vendor/plugins directory of your rails application.

Now go to app/models and open product.rb (i.e the file for your Product model or whichever model you are using). Your model should look something like this:

Code : - fold - unfold

class Product < ActiveRecord::Base
indexes_columns :title, :description, :into=>'idx'
before_save :make_index
before_update :make_index
def make_index
self.idx = self.index_repr
end
end

Now there is a bit of explaining to do here. In addition to all the other columns Product has, you should also add another one called idx which has the type "TEXT" in your SQL database. This means that the columns for product should be id, name, description, price and idx (which is a TEXT column). idx contains the searchable terms for your product. You will notice the line at the top which reads "indexes_columns :title, :description, :into => 'idx' ". Well simple_search simply takes the value of these columns and creates a searchable index which it puts into the idx column. I could well have said "indexes_columns :title, :description, :price :into => 'idx' " which will also include the price as part of our index.

I've added some filters to the product model. Before a product is saved or updated, the idx value is first determined using the 'make_index' method. This saves you the trouble of doing this in your controller each time you save or update your product.

And that is pretty much it. You can fire up script/console with "ruby script/console" for your application.
Create a new product with

Code : - fold - unfold

Product.create(:name => 'ninja_turtle', :description => 'It barks and eats your mother', :price => 20)

Then you can now type Product.find_using_term("ninja") and your recently created product should turn up. Use find_using_term to return any products that match your given term.

Last edited by daibatzu (2006-09-20 09:10:48)

daibatzu · 2006-09-20 05:46:30

Well I've found an even simpler way to implement search. This is unrelated to the iPod shuffle competition by the way (:-> hehe). Um here goes.

First you will need to download the search library by Duane Johnson and Nate McNamara. You can find it here:

Code : - fold - unfold

http://wiki.rubyonrails.org/rails/pages/TextSearch

Copy the code listing at the very end of the website above and save it as a file called search.rb. Place the search.rb file in the lib folder of your rails application. Now lets say you have a model called Product, do the following:

Code : - fold - unfold

#First load the search library in your lib folder
require_dependency "search"
class Product < ActiveRecord::Base
#choose which columns to search
searches_on :all
#anything else your model does
end

searches_on defines what columns I want to be made searchable. I can also say

Code : - fold - unfold

searches_on :title, :description, :list_price

to search only these columns.
Now to search my product, all I have to do is say:

Code : - fold - unfold

Product.search("ninja turtle")

If I need to search for many different queries I should say Product.search("ninja turtle or transformer") but this didn't really work for me because I don't want my users to type strange commands. So this is the search method I'm using:

Code : - fold - unfold

Product.search(params[:query].gsub(' or ',' ').split().join(' or ')

This works for me and should work for you too. This is much simpler than the previous method using simple search.

Last edited by daibatzu (2006-10-06 04:43:11)

tripdragon · 2006-10-06 01:08:30

how did you get it off of the site? I et tons and tons of errors when using copy and paste from just checking that files syntax

daibatzu · 2006-10-06 04:54:42

Mmm, not sure what happened. It was OK previously. You can use the code below but it could be come outdated. The problem I think is in the quotes and double quotes used. Something like ”. Like they're using HTML character codes in Ruby code, maybe to make it look nicer or something? not sure.

Code : - fold - unfold

# Adds search method to ActiveRecord::Base.
# The query language supports the operators
# (), not, and, or
# Precedence in that order.
# - is an alias for not.
# If no operator is present, and is assumed.
# Lastly, anything within double quotes is treated as
# a single search term.
#
# For example,
# ruby rails => records where both ruby and rails appear
# "ruby on rails" => records where "ruby on rails" appears
# ruby or rails => records where ruby or rails (or both) appears
# ruby or chunky bacon => records where ruby appears or both chunky and bacon appear
# not dead or alive => records where alive appears or dead is absent
# -(ruby or rails) => records where neither ruby nor rails appears
# (ruby or rails) -"ruby on rails" => records where ruby or rails appears but not the phrase "ruby on rails"
#
# Query feature by Nate McNamara (nate@mcnamara.net)
# Original TextSearch library by Duane Johnson.
module ActiveRecord
class Base
# Allow the user to set the default searchable fields
def self.searches_on(*args)
if not args.empty? and args.first != :all
@searchable_fields = args.collect { |f| f.to_s }
end
end
# Return the default set of fields to search on
def self.searchable_fields(tables = nil, klass = self)
# If the model has declared what it searches_on, then use that...
return @searchable_fields unless @searchable_fields.nil?
# ... otherwise, use all text/varchar fields as the default
fields = []
tables ||= []
string_columns = klass.columns.select { |c|
c.type == :text or c.type == :string
}
fields = string_columns.collect { |c|
klass.table_name + "." + c.name
}
if not tables.empty?
tables.each do |table|
klass = eval table.to_s.classify
fields += searchable_fields([], klass)
end
end
return fields
end
# Search the model's text and varchar fields
# text = a set of words to search for
# :only => an array of fields in which to search for the text;
# default is 'all text or string columns'
# :except => an array of fields to exclude
# from the default searchable columns
# :case => :sensitive or :insensitive
# :include => an array of tables to include in the joins. Fields that
# have searchable text will automatically be included in the default
# set of :search_columns.
# :join_include => an array of tables to include in the joins, but only
# for joining. (Searchable fields will not automatically be included.)
# :conditions => a string of additional conditions (constraints)
# :offset => paging offset (integer)
# :limit => number of rows to return (integer)
# :order => sort order (order_by SQL snippet)
def self.search(text = nil, options = {})
fields = options[:only] || searchable_fields(options[:include])
if options[:except]
fields -= options[:except]
end
unless options[:case] == :sensitive
text.downcase!
fields.map! { |f| "lower(#{f})" }
end
condition_list = []
unless text.nil?
condition_list << build_text_condition(fields, text)
end
if options[:conditions]
condition_list << "#{options[:conditions]}"
end
conditions = condition_list.join " AND "
includes = (options[:include] || []) + (options[:join_include] || [])
find(:all,
:include => includes.empty? ? nil : includes,
:conditions => conditions.empty? ? nil : conditions,
:offset => options[:offset],
:limit => options[:limit],
:order => options[:order])
end
private
# A chunk is a string of non-whitespace,
# except that anything inside double quotes
# is a chunk, including whitespace
def self.make_chunks(s)
chunks = []
while s.length > 0
next_interesting_index = (s =~ /\s|\"/)
if next_interesting_index
if next_interesting_index > 0
chunks << s[0...next_interesting_index]
s = s[next_interesting_index..-1]
else
if s =~ /^\"/
s = s[1..-1]
next_interesting_index = (s =~ /[\"]/)
if next_interesting_index
chunks << s[0...next_interesting_index]
s = s[next_interesting_index+1..-1]
elsif s.length > 0
chunks << s
s = ''
end
else
next_interesting_index = (s =~ /\S/)
s = s[next_interesting_index..-1]
end
end
else
chunks << s
s = ''
end
end
chunks
end
def self.process_chunk(chunk)
case chunk
when /^-/
if chunk.length == 1
[:not]
else
[:not, *process_chunk(chunk[1..-1])]
end
when /^\+/
if chunk.length == 1
[:and]
else
[:and, *process_chunk(chunk[1..-1])]
end
when /^$.*$$/
if chunk.length == 2
[:left_paren, :right_paren]
else
[:left_paren].concat(process_chunk(chunk[1..-2])) << :right_paren
end
when /^\(/
if chunk.length == 1
[:left_paren]
else
[:left_paren].concat(process_chunk(chunk[1..-1]))
end
when /\)$/
if chunk.length == 1
[:right_paren]
else
process_chunk(chunk[0..-2]) << :right_paren
end
when 'and'
[:and]
when 'or'
[:or]
when 'not'
[:not]
else
[chunk]
end
end
def self.lex(s)
tokens = []
make_chunks(s).each { |chunk|
tokens.concat(process_chunk(chunk))
}
tokens
end
def self.parse_paren_expr(tokens)
expr_tokens = []
while !tokens.empty? && tokens[0] != :right_paren
expr_tokens << tokens.shift
end
if !tokens.empty?
tokens.shift
end
parse_expr(expr_tokens)
end
def self.parse_term(tokens)
if tokens.empty?
return ''
end
token = tokens.shift
case token
when :not
[:not, parse_term(tokens)]
when :left_paren
parse_paren_expr(tokens)
when :right_paren
'' # skip bogus token
when :and
'' # skip bogus token
when :or
'' # skip bogus token
else
token
end
end
def self.parse_and_expr(tokens, operand)
if (tokens[0] == :and)
tokens.shift
end
# Even if :and is missing, :and is implicit
[:and, operand, parse_term(tokens)]
end
def self.parse_or_expr(tokens, operand)
if (tokens[0] == :or)
tokens.shift
[:or, operand, parse_expr(tokens)]
else
parse_and_expr(tokens, operand)
end
end
def self.parse_expr(tokens)
if tokens.empty?
return ''
end
expr = parse_term(tokens)
while !tokens.empty?
expr = parse_or_expr(tokens, expr)
end
expr
end
def self.parse_tokens(tokens)
tree = parse_expr(tokens)
tree.kind_of?(Array)? tree : [tree]
end
def self.parse(text)
parse_tokens(lex(text))
end
def self.apply_demorgans(tree)
if tree == []
return []
end
token = tree.kind_of?(Array)? tree[0] : tree
case token
when :not
if (tree[1].kind_of?(Array))
subtree = tree[1]
if subtree[0] == :and
[:or,
apply_demorgans([:not, subtree[1]]),
apply_demorgans([:not, subtree[2]])]
elsif tree[1][0] == :or
[:and,
apply_demorgans([:not, subtree[1]]),
apply_demorgans([:not, subtree[2]])]
else
# assert tree[1][0] == :not
apply_demorgans(subtree[1])
end
else
tree
end
when :and
[:and, apply_demorgans(tree[1]), apply_demorgans(tree[2])]
when :or
[:or, apply_demorgans(tree[1]), apply_demorgans(tree[2])]
else
tree
end
end
def self.demorganize(tree)
result = apply_demorgans(tree)
result.kind_of?(Array)? result : [result]
end
def self.sql_escape(s)
s.gsub('%', '\%').gsub('_', '\_')
end
def self.compound_tc(fields, tree)
'(' +
build_tc_from_tree(fields, tree[1]) +
' ' + tree[0].to_s + ' ' +
build_tc_from_tree(fields, tree[2]) +
')'
end
def self.build_tc_from_tree(fields, tree)
token = tree.kind_of?(Array)? tree[0] : tree
case token
when :and
compound_tc(fields, tree)
when :or
compound_tc(fields, tree)
when :not
# assert tree[1].kind_of?(String)
"(" +
fields.map { |f|
"(#{f} is null or #{f} not like #{sanitize('%'+sql_escape(tree[1])+'%')})"
}.join(" and ") +
")"
else
"(" +
fields.map { |f|
"#{f} like #{sanitize('%'+sql_escape(token)+'%')}"
}.join(" or ") +
")"
end
end
def self.build_text_condition(fields, text)
build_tc_from_tree(fields, demorganize(parse(text)))
end
end
end

Arghh.. how do I get rid of line numbers.

Last edited by daibatzu (2006-10-06 04:55:50)

daibatzu · 2006-10-24 12:34:55

Ok I got an email asking how to build the search form for this. This is really quite simple and there a number of ways you can do it (including with AJAX). Anyway, here goes.

First, you'll need to add a search action to your controller. This could be like this:

Code : - fold - unfold

def search
if params[:query]
@products = Product.search(params[:query])
else
@products = []
end
end

Next, on any of the views you can add this bit of code. This is basically a search box.

Code : - fold - unfold

<form action='/search'>
<input name="query" type="text" />
<input type='submit' value='Search' />
</form>

Then you create a view called search for your controller. Not the best view but you can use whatever you like and modify it to your liking. So the code below is your search.rhtml.

Code : - fold - unfold

<% if @products.empty? %>
<span>There were no results for your query</span><br/><br/>
<% else %>
<% for product in @products %>
Result: <% product.title %>
<% end %>
<% end %>

So, just a form which accesses the search action in your controller and sends the parameter called query to it. Quite simple.

Last edited by daibatzu (2006-10-24 12:38:10)

tripdragon · 2006-12-05 12:17:38

off hand would you know how to paginate this??

tortoise · 2006-12-18 01:07:12

paginating is pretty simple. Somewhere on the rails wiki I found someone had written a paginate_collection method. Add it to application.rb

Code : ruby - fold - unfold

def paginate_collection(collection, options = {})
default_options = {:per_page => 10, :page => 1}
options = default_options.merge options
pages = Paginator.new self, collection.size, options[:per_page], options[:page]
first = pages.current.offset
last = [first + options[:per_page], collection.size].min
slice = collection[first...last]
return [pages, slice]
end

call it like this in the controller

Code : ruby - fold - unfold

def search
if params[:query]
product_collection = Product.search(params[:query])
else
product_collection = []
end
@product_pages, @products = paginate_collection product_collection, :page => params[:page]
end

then in the view, paginate like this (I separated this stuff into a partial so I can paginate easily)

Code : ruby - fold - unfold

<div class="paginate_controls">
<% if header %>
Showing <%= @product_pages.current.first_item %>
to <%= @product_pages.current.last_item %>
of <%= @product_pages.item_count %><br/>
<% end %>
<%= link_to(h('< Previous'), {:page => @product_pages.current.previous}) + " | " if @product_pages.current.previous %>
<%= pagination_links(@product_pages, :window_size => 4) %>
<%= " | " + link_to(h('Next >'), {:page => @product_pages.current.next}) if @product_pages.current.next %>
</div>
<div class="search_results">
<% @products.each do |product|
<%= product.name %>
<% end %>
</div>

tortoise · 2006-12-21 22:28:17

I should note that the above paginating does not work for searches, because when you go to page two the search query defaults to nothing and you lose all your search results. Haven't figured out a clean solution to that just yet.

paron · 2007-01-11 19:24:03

Ok, I posted a tutorial on paginating search results (with semantically-meaningful labels, yet) at http://wiki.rubyonrails.com/rails/pages … teSearches Let me know what you think.

Last edited by paron (2007-01-24 13:08:57)

nazrul · 2007-01-23 11:45:28

Great tutorial! How to search by category ? I mean if I'm developing my school library website, how I want to let the students search by by Category, Year, Author ? Like this forum search options. Thanks!

paron · 2007-01-24 12:47:07

In the tutorial, the model provides two methods, "search_name" and "search_all."

For what you're trying to do, I would just provide one search method -- "search_author". I suspect search is overkill for the other two conditions, "category" and "year".

There's probably no need to "search" categories; you already know what they are. It's more of a filter, where you might pass an argument like ":conditions=>["category = 'fiction']. It's the same with year; you really don't need to 'search' it -- you're just going to filter for "1987" or ">1994 AND <2004". You're not going to search in the "year" field for terms like "Einstein" AND "math", so it isn't a search, per se.

Anyway, once your model returns its array of results, you could paginate and semantically label the pages with author data, so that the results are neatly broken into pages labeled "Aaron to Byron", "Camille to Dante", etc.

If it's a multi-year search, you could label them with "Able, 1883 to Anderson, 1994" and "Astarte, 1776 to Ballinger, 1923" or the other way around, like: "1810, Pursley to 1850, Bedinger" and "1851, Sydney to 1876, Lincoln" depending on your ordering of the results.

Is that what you're asking?

Ron

Last edited by paron (2007-01-24 12:47:43)

seandick · 2007-09-14 19:11:37

remove line numbers in vim: %s/^.\{,3}\d\{,3}\W//g

steveroot · 2008-03-03 19:52:06

RoR newb here,
In suggested search.rhtml I found it didn't display the results. Eventually I found I had to change
Result: <% product.title %>

into

Result: <%= product.title %>

kingworks · 2008-03-19 14:41:35

I hope this thread hasn't been dead and buried too long.

I am having trouble with the search.rb (the second version presented) file generating a syntax error. The problem is identified as being line 245:

Code : - fold - unfold

244 def self.parse_and_expr(tokens, operand)
245 if (tokens[0] :and)
246 tokens.shift
247 end
248 # Even if :and is missing, :and is implicit
249 [:and, operand, parse_term(tokens)]
250 end

But line 253, which is similar, is identified, too:

Code : - fold - unfold

252 def self.parse_or_expr(tokens, operand)
253 if (tokens[0] :or)
254 tokens.shift
255 [:or, operand, parse_expr(tokens)]
256 else
257 parse_and_expr(tokens, operand)
258 end
259 end

Any help is greatly appreciated.

EDIT: It appears that every instance of ":and" and " yikes

r" is generating a syntax error - also appear on lines 293 and 297 of search.rb, respectively.

Last edited by kingworks (2008-03-19 14:46:43)

jerinantony1 · 2008-10-10 08:33:13

hi all i hv tried out this tutorial but getting an error

-->"undefined method `search' for #<Class:0x467cd08>"

c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:1672:in `method_missing'
app/controllers/book_controller.rb:65:in `find'

#1 2006-09-08 00:24:55

#2 2006-09-20 05:46:30

Re: HOWTO: Adding search to your rails application

#3 2006-10-06 01:08:30

Re: HOWTO: Adding search to your rails application

#4 2006-10-06 04:54:42

Re: HOWTO: Adding search to your rails application

#5 2006-10-24 12:34:55

Re: HOWTO: Adding search to your rails application

#6 2006-12-05 12:17:38

Re: HOWTO: Adding search to your rails application

#7 2006-12-18 01:07:12

Re: HOWTO: Adding search to your rails application

#8 2006-12-21 22:28:17

Re: HOWTO: Adding search to your rails application

#9 2007-01-11 19:24:03

Re: HOWTO: Adding search to your rails application

#10 2007-01-23 11:45:28

Re: HOWTO: Adding search to your rails application

#11 2007-01-24 12:47:07

Re: HOWTO: Adding search to your rails application

#12 2007-09-14 19:11:37

Re: HOWTO: Adding search to your rails application

#13 2008-03-03 19:52:06

Re: HOWTO: Adding search to your rails application

#14 2008-03-19 14:41:35

Re: HOWTO: Adding search to your rails application

#15 2008-10-10 08:33:13

Re: HOWTO: Adding search to your rails application

Board footer