Question

I have two files (which could be up to 150,000 lines long; each line is 160 bytes), which I’d like to check to see if the lines in each are the same. diff won’t work for me (directly) because a small percentage of the lines occur in a different order in the two files. Typically, a pair of lines will be transposed.
What’s the best way to see if the same lines appear in both files, but where order doesn’t matter?
Thanks,
Chris

Answer 1

Although it’s a slightly expensive way to do it (for anything larger I’d rethink this), I’d fire up python and do the following:

filename1 = "WHATEBVER YOUR FILENAME IS"
filename2 = "WHATEVER THE OTHER ONE IS"
file1contents = set(open(filename1).readlines())
file2contents = set(open(filename2).readlines())
if file1contents == file2contents:
    print "Yup they're the same!"
else:
    print "Nope, they differ.  In file2, not file1:\n\n"
    for diffLine in file2contents - file1contents:
        print "\t", diffLine
    print "\n\nIn file1, not file2:\n\n"
    for diffLine in file1contents - file2contents:
        print "\t", diffLine

That’ll print the different lines if they differ.