I’ve got a gaming-oriented website with 200+ users. The site has a large database tracking user plays, and one of the motivations for continued participation is the extensive statistics and rankings (S&R) with which the site provides the user.
As the list of S&Rs tracked has grown, some of the more intricate calculations have been moved to tables within the database, rather than be generated on-the-fly in order to improve page loading speed.
However, I plan to move from extensive to exhaustive S&Rs by the end of the year, increasing the overall number of datapoints available to the user by a factor of 10. I’ve already decided to stop doing on-the-fly queries and to move all the calculations to a cron job, but I’m unsure where to store the data.
Given a user base <1000, would it make more sense to place this data within the database or read/write a text file for each user’s stats?
These are the main pros and cons in my mind:
Storing S&Rs in the Database
+ cross-user comparisons are easy and fast
+ faster cron jobs because there's no need to write to many, many files
- database table count will jump from ~50 to 200+ (at least)
- one point of failure (database corruption) for all site data
- modifying S&R structure requires modifying database as well
Storing S&Rs in Text Files
+ neatly organized and distributes data corruption risk
+ database is easier to navigate
+ redesigning S&R structure is done by simply modifying script and
overwriting all text files, rather than adjusting database tables
- cron job will have to read/update XXX files each time
- cross-user comparisons are difficult and time-consuming
But I’ve never done something of this magnitude before, so I’m not really sure (for example) if a 200+ table MySQL database is even really a problem?
I’d appreciate any suggestions you can provide! 🙂
Any popular database software should be able to handle millions of entries, having 200+ tables is not an issue on that end.
Corruption is unlikely, but on a site of that nature you should be doing backups fairly frequently, and preferable storing a copy outside the server – using individual files distributes and decreases the likelihood of a general failure, but there’s also a small chance of problems ocurring.
Database software excels at performing tasks on it’s data, using flat files would probably force you to write your own method to process them, and this could easily prove to be a major task, at the extra cost of a loss of speed compared to using a database (I’m just assuming this, I might be very wrong).