Question

I have 3 separate servers with a sharded image directory (a0-z9). Unfortunately my rsync and php upload script broke along the way and now I have 3 unmatched directories on 3 servers that are supposed to be identical. I’m not concerned with overwriting… if it’s already there, there’s no need to overwrite it — it’s the same.

ServerA

/path/img/a0/001/
/path/img/a0/.../
/path/img/a0/999/
/path/img/../
/path/img/z9/001/
/path/img/z9/.../
/path/img/z9/999/

ServerB and ServerC have the same layout. All have the exact same 2-digit directory structure (260 directories, a0-z9). But they have different sub-directories. ServerA may have a0/001/, but this directory might be missing on servers B & C (or the other way around).

I need to merge all 3 directories.

Ultimately if c7/003/ exists on one server, it needs to exist on all 3.

How can I do this? I’ve already fixed it going forward, but I still need to clean up the current mess.

Answer 1

The rsync command for syncing two folders (given a good source and an outdated/broken destination) is provided below, your script dies probably because it runs too long. Try running it for smaller subdirectories.

rsync -a --delete-after /source/images/ -e 'ssh -l username' \
    hostname:/path/to/destination/images/