Question

I have a main database that stores up to 5’000 new rows per day.

I want to have a second database that only contains the latest 30 days worth of data at any time.

Therefore I plan to set up a cron job that regularly dumps the rows older than 30 days and copies the new ones.

What would be the best design for the copying part?

• Copying on the fly with MySQL alone
• An MySql export to a txt file, then an MySql import, then deleting the temporary file
• A php script that iterates through the rows and copies them one by one

I want robustness and minimum amount of CPU/memory usage

Answer 1

The quickest and most robust way is to perform the transfer directly in MySQL. Here are the steps involved:

First, create the second table:

CREATE TABLE IF NOT EXISTS second.last30days LIKE main_table;

Next, insert the records 30 days old, or newer:

INSERT INTO 
second.last30days
SELECT
*
FROM
main_table
WHERE
created >= CURDATE() - INTERVAL 30 DAYS
ORDER BY created;

Lastly, delete the records older than 30 days:

DELETE FROM 
second.last30days
WHERE
created < CURDATE() - INTERVAL 30 DAYS
ORDER BY created;

It would be advisable to not run the INSERT and DELETE statements at the same time.