Question

I have two identical (in structure) databases residing on separate backend servers.

I need to come up with some logic to ‘merge’ their data into a single database on a third server.

My initial design is to load their data (by table) into memory using a combination of Perl hashes and arrays and merging them there, then doing a single massive write to a local DB (also identical in structure).

I would repeat for all tables (4-5).

I’ve seen posts about merging tables, but not sure if I can use some of those responses as my tables reside in separate databases (let alone separate machines).

My question is am I stuck with having to load the results into memory first or are there features of MySQL that I can use to my advantage?

Answer 1

What “mu” said needs addressing, but I’m not sure I’d go with this approach at all.

1. Get the two databases onto the target server using standard mysql dump/restore
2. Use standard queries to merge them into the third DB using standard queries

You should let MySQL do the heavy lifting.