Question

I’m using solr via the sunspot gem in a rails project.

I am indexing scraped data.

My indexing is currently done like so:

searchable do
  text :title, :boost => 3.0 do
    title.gsub(/\'s\b/, "")
  end
  text :mentions do
    mentions.map do |mention|
      mention.title.gsub(/\'s\b/, "")
    end
  end
end

Currently, if I do:

Video.solr_search { fulltext '"Radiohead"' }

Solr will return results with:

Radiohead's

and

Radiohead

I would like to only find:

Radiohead

Is there a way to do this via Sunspot?

Answer 1

Check what filters you have defined in the analyzer section of the field type for your field in schema.xml (in …/solr/conf directory). Here’s an example:

  <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
        <analyzer type="index">
            ...
            <filter class="solr.SnowballPorterFilterFactory" language="English" />
        </analyzer>
  </fieldType>

The behaviour you’re seeing is called “stemming” – it’s where the indexed value is the stem of the word, rather than the word itself. eg, “fly”, “flies”, “flew” and “flying” would all be indexed as “fly”. If there’s a filter like snowball (apache’s stemmer), then you’ll get the behaviour you’re seeing. Try removing the filter, restarting solr then reindexing your documents.