Home/ Questions/Q 7761443
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-01T14:08:14+00:00

How to find potentially duplicate files in a given directory? Is there some existing

  • 0

How to find potentially duplicate files in a given directory? Is there some existing tools for this sort of thing? Some heuristics I can think of:

  • it should be recursive.
  • it should only compare file names and size and nothing else
  • it should be able to find duplicates when the file names are trivially different such as “foobar.txt” and “foobar.txt (2)”
  • I have the files on a drive that can be mounted to Linux, Mac OS X, or Windows, as desired
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Under Linux/UNIX, you can use the “sum” or “md5cum” command to generate a checksum for each file. Then just look for files with the same checksum.

    The superquick way of doing this would be to sort the output of the sum command with the “sort” command and look for consecutive entries with the same checksum.

    The superquick way to get a list of the duplicate files would be sort the output of the sum command to a file, then do it again with the “-u” parameter to sort, making a unique list, and diff the files. The difference will be the duplicate files.

    • 0