Question

A system generates transaction records and puts them into a RMDB table even if they are failure cases. This table will be queried at the real time and also for generating reports at the end of a day.

This table will no doubt grow fairly big quickly. Is this a problem? Will real-time queries run really slowly if the table is large? What are the solutions?

Answer 1

Generally it’s not a great idea to have your production database double as your reporting database. There are several reasons for this. One of them is that often times, it doesn’t make sense for your report DB schema to be the same as your production DB. Normally your reporting DB will be somewhat more denormalized than your production DB, for reporting performance reasons.

Another reason is that the indexing requirements of your reporting database will normally be different than your production database. Your reporting database will usually have more indexes than your production database. You typically don’t need as many indexes for production purposes, and you don’t normally want them. In a heavy transactional system, large number of indexes can cause performance issues.

Also, it’s typically not necessary to keep as much data in your production DB as in your reporting DB. At our company, we keep just enough data in our production DB to satisfy production needs. This is typically 12 months. In our reporting DB, we have data going back much further. Also, it’s important to try and keep your production DB as small as possible for disaster recovery purposes. The larger your DB, the longer it takes to restore, and the longer time you’re offline.