Home/ Questions/Q 498257
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-13T05:51:40+00:00

I am using difflib to compare files in two directories (versions from consecutive years).

  • 0

I am using difflib to compare files in two directories (versions from consecutive years).
First, i am using filecmp to find files that have changed and then iteratively using difflib.SequenceMatcher to compare them and generate a html diff as explained here.

However, I find that the program is taking too long to run and python is utilizing 100% CPU. On time profiling, i found that the seqm.get_opcodes() call which is taking all the time.

Any insight would be appreciated.
Thanks !

Code:

#changed_set contains the files to be compared
for i in changed_set:
  oldLines = open(old_dir +"/" + i).read()
  newLines = open(new_dir +"/" + i).read()
  seqm = difflib.SequenceMatcher(lambda(x): x in string.whitespace, oldLines, newLines)
  opcodes = seqm.get_opcodes() #XXX: Lots of time spent in this !
  produceDiffs(seqm, opcodes)
  del seqm
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    My answer is a different approach to the problem altogether: Try using a version-control system like git to investigate how the directory changed over the years.

    Make a repository out of the first directory, then replace the contents with the next year’s directory and commit that as a change. (or move the .git directory to the next year’s directory, to save on copying/deleting). repeat.

    Then run gitk, and you’ll be able to see what changed between any two revisions of the tree. Either just that a binary file changed, or with a diff for text files.

    • 0