Question

I have a fairly simple table with approx a million rows.

id  |  my_col  |  other1  |  other 2 | ...

There are about 15k distinct my_col values in this table and I have an index on my_col. I have a set of 7k my_col values that I need to remove from this table.

What is the more efficient thing to do in SQL (I’m currently working with MySQL, but may port to MS SQL in future).

Is it a) In my java app code, itterate through all the my_col values and call sql delete on each one.

for (String my_colValue : listMyCol) {
   [delete from my_table where my_col = my_colValue]
}

or b) Build up a single SQL [large] statement containing all these values using the “where in” clause ?

delete from my_table where my_col in ('aaa', 'aab', 'aac', ...)

I’d guess it is b), but I’m not sure if specifying about 7k values in this “where in” clause becomes inefficient.

For what its worth, my app server and database server are both hosted in Amazon, but on separate tiers.

Answer 1

c) Recreate your table.

You are going to delete half of you rows, so think about it.
While a) and b) will take maybe forever, recreate your table will be tricker but immediate.

And you need to load your 7k values in a temporary tables, then it is easy :

CREATE TABLE newMyTable
AS
SELECT myTable.*
FROM myTable
    INNER JOIN myValues
        ON myTable.my_col = myValues.my_col

Or if you can’t create table, maybe this will be fast enough :

DELETE FROM myTable t
WHERE EXISTS
(
    SELECT *
    FROM myValues v
    WHERE t.my_col = v.my_col
)

But the only things you need to keep in mind : you have to create a table with your 7k values.