Question

Let’s say you have a random number generator spitting out numbers between 1 and 100 000 000 and you want to store them in a database (MySQL) with the timestamp when they were generaeted. If a number that has previously been seen comes, it is discarded.

What would be the best algorithm to make this happen? SELECT then INSERT as necessary? Is there something more efficient?

Answer 1

1. You can go for a SEQUENCE:

   +

   • no relations are being locked, thus best performance;
   • no race conditions;
   • portable.

   -

   • it is possible to get “gaps” in the series of numbers.

2. You can do a SELECT ... then INSERT ...:

   +

   • no gaps, you can also do some complicated math on your numbers.

   -

   • it’s possible to get another parallel session in the middle between SELECT and INSERT and end up with 2 equal numbers;
   • if there’s a UNIQUE constraint, then previos situation will lead to an exception;
   • to avoid such situation, you might go for an explicit table locks, but this will cause an immediate performance impact.

3. You can choose INSERT ON DUPLICATE KEY UPDATE, and by now it seems to be the best option (take a look at "INSERT IGNORE" vs "INSERT … ON DUPLICATE KEY UPDATE"), at least in my view, with the only exception — not portable to other RDBMSes.

P.S. This article is not related to MySQL, but it is worth reading it to get an overview of all the catches that can happen on your way.