I have been tasked with refactoring a project that currently uses an EAV model for data storage.
In some implementations, the main table holds several million rows. As you can imagine, I consider this an abomination.
The refactor is driven by a need to do spatial searches, which don’t really work when your lat and long are stored as comma separated text in a VARCHAR field… but I digress.
The nature of the project means that producing a static set of normalised tables in advance, is out of the question. Think of it as a kind of a dynamic form builder, where the number of fields in a new form is unknown, the number of options for a dropdown is unknown, etc.
I’m toying with the idea of creating a “DynamicTable” class, that would maintain a dictionary of fields, and manage a dynamically created set of tables. Caching is already very well implemented (using memcached) so the dictionary would only need to be queried when the schema is changed (rare after initial commit) and on reboot (even more rare).
My initial attempts at fleshing out even something basic are making me realise the full scale of what I want to do, and my searches for similar projects have been pretty fruitless.
Can anyone:
- Commend mycurrent approach and wish me luck.
- Recommend an alternative approach that is better, even if it isn’t necessarily easier.
- Recommend any projects I can look at that achieve a similar goal (my searching yields nothing!)
Cheers guys!
Answering my own question then.
Taken the advice of Charles I investigated several NoSQL Document DB options, and eventually decided on MongoDB.
It’s an excellent fit for the type of data I am trying to store, and so very easy to work with.