Home/ Questions/Q 8672827
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-12T19:18:00+00:00

I have a table with 15,000,000 records. These records can be related to one-another

  • 0

I have a table with 15,000,000 records. These records can be related to one-another based on certain matching fields (typically a max of 3 or 4 in a single group). Each record also has a date associated with it, however this date does not necessarily correlate to the date/order it was loaded into the table.

These records are periodically used to updated other records in the database. I only care about the LATEST record in a single group, though, and at 15 million records (which are updating another 10 million records), I want to make sure that I ignore everything else. I also want to skip an update if the latest record already has an “updated” flag.

I’ve been using the a view like this to collect the latest record within each group:

SELECT a, b, c, max(scan.timestamp) AS latest_scan
FROM scan
GROUP BY a, b, c;

I then use a WHERE clause in my UPDATE to ignore any records with the updated flag.

With this many records, am I okay using a view? I’ve seen people start creating real indexible tables on the fly in order to increase performance, but I still feel like that would take ages. Is this something that I could benefit from?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Mostly you need to try it and see what the performance is like.

    The key thing is that the view will end up being a quick notation of a sub-query which will give the optimizer information. So, it’s still going to mostly depend on whether the optimizer can find indexes to use for the SQL statements you call the database with.

    Notice that you probably don’t need the ORDER BY in your view, since you will be using it in some other SQL statement, which can then order the records. If the optimizer doesn’t figure out that it can skip the order by, then you’re just going to spend time sorting when you won’t need to.

    I would expect you could use a GROUP BY a, b, c to get the max timestamp for the group.

    • 0