Question

Situation: our production mysql database makes a daily dump into a .sql file. I’d like to keep a shadow database that is relatively up to date.
I know that to create a mysql database from a .sql file, one uses:

mysql -u USERNAME -p DATABASENAME < FILE.SQL

For our db, this took 4-5 hours. Needless to say, I’d like to cut that down, and I’m wondering if there’s a way to just update the db with what’s new/changed. On Day 2, is there a way to just update my shadow database with the new .sql file dumped from the production db?

Answer 1

MySQL Replication is the way to go.

But, in cases, where that is not possible, use the following procedure:

Have a modifed timestamp column in all your tables and update this value whenever a row is inserted/changed.

Use the following mysqldump options to take the incremental SQL file (this uses REPLACE commands instead of insertcommands, since the existing record will be updated in the backup database).

Keep a timestamp value somewhere placed in the file system. and use it in the where condition. MDFD_DATE is the column name on which you need to filter. On successful backup, update the value stored in the file.

skip-tz-utc prevents MSQL from automatically adjusting the timestamp values, based on your timezone.

mysqldump –databases db1,db2 –user=user –password=password –no-create-info –no-tablespaces –replace –skip-tz-utc –lock-tables –add-locks –compact –where=MDFD_DATE>=’2012-06-13 23:09:42′ –log-error=dump_error.txt –result-file=result.sql

Use the new sql file and run it in your server.

Limitations:

This method will not work if some records are deleted in your database. You need to manually delete them from the backup databases. Otherwise, keep a DEL_FLAG column and update it to ‘Y’ in production for deleted records and use this condition to delete records in the backup databases.