Question

There are two LAMP servers, A and B, in different countries.
Once a day A has an updated set of data, that is to be processed on B.
What would be the best way to set a communication between them?

Estimated size of an update is 20Mb compressed.

Options that comes to my mind are:

1. curl POST request, uploading compressed data in JSON;
2. SQL dump of a tmp table, compressed, then rsync’ed to B. B watching specific folder for update. Then importing the table and processing;
3. Message Queuing with RabbitMQ or Amazon Simple Queue;
4. ..other option?

Should you have any experience with a similar tasks, please share your thoughts.

Answer 1

option 2, either if they are on a network or even using ftp, you can use cronjobs to run tasks at scheduled hours (probably during the night).