Home/ Questions/Q 3939562
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-20T00:18:52+00:00

I’m curious if anyone noticed any scaling issues with Sunspot-Solr . Even if I

  • 0

I’m curious if anyone noticed any scaling issues with Sunspot-Solr . Even if I remove all the searchable params, and its just counting against the raw class by itself; it still takes 5 to 8 seconds to load on my local, 4 to 5 seconds on production.

Has anyone else been able to scale Sunspot-Solr ? What are some common problems?

How can one look into this deeper?

Here’s the Solr Logs for a single request :

Solr Select (208.1ms)   {:rows=>20, :start=>0, :q=>"*:*", :sort=>"score desc", :fq=>["type:Organization", "published_b:true", "updated_at_d:[2009\\-02\\-03T16\\:11\\:55Z TO *]"]}

Solr Select (5.5ms)   {:rows=>20, :start=>0, :q=>"*:*", :sort=>"score desc", :fq=>["type:Organization", "published_b:true", "updated_at_d:[2009\\-02\\-03T16\\:11\\:55Z TO *]"]}

Solr Update (12.6ms)   <?xml version="1.0" encoding="UTF-8"?><add><doc><field name="type">User</field><field name="type">ActiveRecord::Base</field><field name="id">User 2</field><field name="class_name">User</field><field name="first_name_s">Bob</field><field name="created_at_d">2009-09-28T21:00:27Z</field><field name="last_name_s">Marley</field><field name="email_s">bob.marley@gmail.com</field><field name="name_s">Bob Marley</field><field name="last_name_text">Marley</field><field name="first_name_text">Bob</field><field name="email_text">bob.marley@gmail.com</field><field name="name_text">Bob Marley</field></doc></add>


Solr Update (487.7ms)   <?xml version="1.0" encoding="UTF-8"?><commit/>
Completed in 12632ms (View: 11633, DB: 228) | 200 OK [http://localhost/organizations/search]
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    1000 objects is child’s play for Solr, so there’s something fishy going on here with the ~200ms Solr read. However, your most immediate problem is that you’re writing to Solr during what appears to be a GET request — what’s up with that? Are you saving a searchable object, which is triggering Sunspot’s auto-index? If you have the need to update models during the course of a GET request (which should probably be done in a background job if possible), you’ll want to disable auto-indexing in Sunspot:

    searchable :auto_index => false
  # sunspot setup
end

    Then you’d need to explicitly call my_model.index in your controllers when you actually do want to update them in Solr.

    Finally, that big update at the end is a Solr commit, which tells Solr to write unstaged changes to disk and load up a new searcher that reflects those changes. Commits are expensive; Sunspot::Rails by default performs a commit at the end of any request that writes to Solr, but this behavior is targeted at the principle of least surprise for new users of Sunspot rather than a live app in production. You’ll want to disable it in your config/sunspot.yml:

    auto_commit_after_request: false

    You’ll then probably want to configure autoCommit in your solr/conf/solrconfig.xml — it’s commented out in the default Sunspot Solr distribution, and there is an explanation in there too. I’ve found that once a minute is a good place to start.

    After making those changes, I’d see if your reads are still slow — I think it’s quite possible that the cause of that is that every time you search, your write/commit to Solr is causing it to have to load up a fresh searcher from disk. So, it can’t allow any of its internal caches to warm, etc., and is generally under a tremendous amount of strain.

    Hope that helps!

    • 0