Our website needs to give out data to the world. This is open-source data that we have stored, and we want it to make it publicly available. It’s about 2 million records.
We’ve implemented the search of these records using Lucene, which is fine, however we’d like to show an individual record (say the user clicks on it after the search is done) and provide more detailed information for that record.
This more detailed information however isn’t stored in the index directly… there are like many-to-many relationships and we use our relational database (MySQL) to provide this information.
So like a single record belongs to a category, we want the user to click on that category and show the rest of the records within that category (lots more associations like this).
My question is, should we use Lucene also to store this sort of information and retrieve it through simple search (category:apples), or should MySQL continue doing this logical job? Should I use Lucene only for the search part?
EDIT
I would like to point out that all of our records are pretty static…. changes are made to this data once every week or so.
Lucene’s strength lies in rapidly building an index of a set of documents and allowing you to search over them. If this “detailed information” does not need to be indexed or searched over, then don’t store it in Lucene.
Lucene is not a database, it’s an index.