I plan to develop a web application for which I need to decide the “type” of my database system. I plan to develop the web app in Django/Python.
My application would contain just one table with ‘n’ increasingly growing (order of millions) records and ‘k’ columns (1-100). Unfortunately, I have no control over the number of ‘k’ viz. the size of ‘k’ may increase/decrease over time (between 1-100 though). I wouldn’t like to have a trade off on performance.
What type of database should I use ? My initial reactions are as follows :
- MySQL : I would like to avoid this option
- Redis : Nice option but the (key,value) store doesn’t fit my bill
- XML databases : Looks good for the task but does it give the desired performance?
- Object Oriented Relational DB : I am not sure on this
Edit
Specific Requirement :
To perform a lookup on the k-columns and retrieving a subset of ‘n’ records. I plan to provide a REST api similar to Twitter or Facebook.
For example, http://api.twitter.com/1/users/show.json?screen_name=Ashton . If this was what I am planning to implement the screen_name would be one of the columns. Similarly, there could be other parameters based on other columns like user_id, number of friends etc.
It sounds like you might be coming into this decision with too many preconceptions. Why do you want to avoid MySQL? Is it because you’re avoiding rigid schema?
Redis doesn’t sound like a fit because of data volume, but I don’t think it has anything to do with it being a key/value store.
Your data needs sound like a pretty textboook case for MongoBD. Have you looked at it?