Question

I’m dealing with a database for about 25,000 users who add about 6 rows on average every day (employees keeping logs for work orders). Basically the database grows indefinite and contains millions of rows (to divide among these 25,000 users).

After a user logs in, I would like the system to display some of their totals such as miles driven in truck number xyz for their entire work career, total time worked on order item xyz and so on. Basically, every time a user logs in, these totals need to be present instantly. In addition, once a user adds a row for a work order, the totals need to reflect this change instantly.

Is it advised to build a totals table per user that gets updated with every entry. Or should I just query the database and have it calculate the total on the fly each time a user logs in (no total tables). Would that however create a bottleneck if users log in every second and the database needs to spit out a total based on millions of rows? How does google do it? 🙂

Thanks.

Answer 1

You might find that a simple query is fast enough with an appropiate index (e.g. index user_id). This should reduce the number of rows that need to be scanned.

But if this is not fast enough, you could calculate the result for all users overnight, and cache this result in another table. You can then do the following:

• Get the total up to the last cache update directly from the cache table.
• Get the total since the last cache update from the main table.
• Add these two numbers to get the overall total.

Another option is to use triggers to keep the pre-calculated result accurate, even when rows are inserted, updated or deleted.