Question

I having been working on an SVN repo using command line only. I now have to bring in users that require a GUI to interface with the repo, however this is presenting a number of problems with similarly named files.

As it so happens a large number of images have been duplicated for reasons due to lack of communication or laziness.

I would like to be able to search for all files recursively from a given folder, and identify all files that differ only by case/capitalization, and must have the same file size, as it is certainly possible conflicts exist between different files, although I’ve not encountered any yet.

I don’t mind to hammer out a Perl script to handle this myself, however I’m wonder if such a thing already exists or if anybody has any tips before I roll my sleeves up?

Thanks 😀

Answer 1

I lean on md5sum for this type of problem:

find * -type f | xargs md5sum | sort | uniq -Dw32

If you are using svn, you’ll want to exclude your .svn directories. This will print out all files with their paths that have identical content.

If you really want to only match files that differ by case, you can add a few more things to the above pipeline:

find * -type f  | xargs md5sum | sort | uniq -Dw32 | awk -F'[ /]' '{ print $NF }' | sort -f | uniq -Di
myimage_23.png
MyImage_23.png