Imagine you have a web application written in Django and Python 2.65, and MySQL 5.1 is your database of choice.
Now, imagine you will need to scale your app to handle searching 100’s of thousands of document and potentially 100’s of thousands of users will be using it.
Reality: Haystack 1.0 with PySolr and Solr 1.4.0 is proving slow in the above scenarios. Is MyISAM a more workable alternative or should I spend more time working with my current configuration using Solr in a “smarter” way?
Suggestions? Tips?
Thank you for any help!
Michaux
I assume you’re talking about comparing Solr vs MySQL full-text search, otherwise it would be comparing apples to oranges.
I don’t know about Haystack or PySolr, but Solr itself should have no problems handling documents in the order of 100000 with lots of users. Those two parameters alone are not enough to spec the problem, though. For example, frequency of updates, real frequency of requests, document size, page size, sorting, faceting, etc.
Solr is easily scalable, both vertically and horizontally, and is Apache-licensed, while the horizontal scaling solution for MySQL is GPL+commercially licensed.
I disagree with Badger’s answer about Tomcat, it’s a very polished, proven, stable server that’s been around for over 10 years, and the Java performance myth has to be abolished once and for all.
Bottom line: it’s very likely that you need to optimize your Solr instance (both the client-side querying and the server-side index and performance settings). Solr powers some of the biggest websites so it’s quite likely that it can handle your load as well.