Question

I’m trying to write a bash shell script to sync content on two different paths.

The algorithm I’m striving for consists of the following steps

1. given two full (as opposed to relative) paths
2. recursively compare files (whose filename optionally may have
   basename and suffix) in corresponding directories of both paths
3. if either corresponding directories or files are not present, then
   copy each file (from the path with the folder) to the other
   corresponding folder.

I’ve figured out steps 1 and 2 which are

OLD_IFS=$IFS
# The extra space after is crucial
IFS=\

for old_file in `diff -rq old/ new/ | grep "^Files.*differ$" | sed 's/^Files \(.*\) and .* differ$/\1/'`
do
   mv $old_file $old_file.old
done
IFS=$OLD_IFS

Thanks.

Answer 1

I have implemented a similar algorithm in Java, which essentially boils down to this:

1. Retrieve a listing of directories A and B, e.g. A.lst and B.lst

2. Create the intersection of both listings (e.g. cat A.lst B.lst | sort | uniq -d). This is the list of files you need to actually compare; you will also have to descend to any directories recursively.

   You may want to have a look at the conditional expressions supported by your shell (e.g. for bash) or by the test command. I would also suggest using cmp instead of diff.

   Note: you need to consider what the proper action should be when you have a directory on one side and a file on the other with the same name.

3. Find the files that are only present in A (e.g. cat A.lst B.lst B.lst | sort | uniq -u) and copy them recursively (cp -a) to B.

4. Similarly, find the files that are only present in B and copy them recursively to A.

EDIT:

I forgot to mention a significant optimization: if you sort the file lists A.lst and B.lst beforehand, you can use comm instead of cat ... | sort | uniq ... to perform the set operations:

• Intersection: comm -12 A.sorted.lst B.sorted.lst

• Files that exist only in A: comm -23 A.sorted.lst B.sorted.lst

• Files that exist only in B: comm -13 A.sorted.lst B.sorted.lst