Question

I have a MySQL database with about 30,000 rows. I update this database to a remote server nightly, but never are more than 50 rows updated at a time. I am the only one who updates the database. I would like to develop a method in which only CHANGED rows are exported to the remote server.

To save space in the database and to save time when I export to the remote server, I have built “archive” tables (on the remote server) with records that will no longer be updated, and which do not reside on the local database. But I know splitting up this data into multiple tables is bad design that could lead to problems if the structure of the tables ever needs to be changed.

So I would like to rebuild the database so that ALL the records with similar table structures are in a single table (like they were when the database was much smaller). The size of the resulting table (with all archived records) would exceed 80,000 rows, much to large to export in a whole-database package.

To do this, I would like to
(1) Update a “last updated” timestamp in each row when the row is added or modified
(2) Select only rows in tables for export when their “last update” timestamp is greater than the timestamp of the last export operation
(3) Write a query that builds the export .sql file with only new and updated rows
(4) Update the timestamp for the export operation to be used for comparison during the next export

Has anyone ever done this? If so, I would be grateful for some guidance on how to accomplish this.

Steve

Answer 1

If you add a column with a timestamp datatype, for example last_updated timestamp, it will be automatically updated to now() every time a row changes.

Then early every day, simply ship yesterday’s changes:

select * from mytable
where last_updated between subdate(CURDATE(), 1) and CURDATE()