Question

I’m trying to figure out a single sql statement to remove duplicate records from a table but allowing up to 10 duplicates.

I can do the following query to find these duplicates, but I have no idea what query to do to remove all but the first 10 duplicates:

SELECT 
    ip,
    date,
    count(ip) as count
FROM 
    table 
WHERE 
    date 
        BETWEEN 
            '2012-02-03 00-00-00' 
        AND 
            '2012-02-03 23:59:59' 
group by 
    ip
having 
    count > 10

I’m not sure if this is even possible, can someone help me out?

Answer 1

Try this:

delete t2 
from table t2
join (
    SELECT ip, date, (
            select count(*) from 
            table t1 
            where t1.ip = t.ip and t1.date <= t.date and 
                t1.date BETWEEN '2012-02-03 00-00-00' AND '2012-02-03 23:59:59'
        ) as row 
    FROM table t
    WHERE date BETWEEN '2012-02-03 00-00-00' AND '2012-02-03 23:59:59'
) t3 on t2.date = t3.date and t2.ip = t3.ip and t3.row > 10

It uses some kind of sequentially row numbering, please mind that it’s going to work only if pair date and ip is unique.