Question

Is there a reliable, automatic way (such as a command-line utility) to check if two Python files are equivalent modulo whitespace, semicolons, backslash continuations, comments, etc.? In other words, that they are identical to the interpreter?

For example, this:

import sys
sys.stdout.write('foo\n')
sys.stdout.write('bar\n')

should be considered equivalent to this:

import   sys
sys.stdout.\
    write('foo\n'); sys.stdout.\
    write(

    'bar\n') # This is an unnecessary comment

Answer 1

Use the ast module.

Example (for Python 2):

import ast

x = r'''import sys
sys.stdout.write('foo\n')
sys.stdout.write('bar\n')'''

y = r'''import   sys
sys.stdout.\
    write('foo\n'); sys.stdout.\
    write(

    'bar\n') # This is an unnecessary comment'''

xd = ast.dump(ast.parse(x))
yd = ast.dump(ast.parse(y))
print xd == yd

You can of course read in the source code from actual files instead of string literals.

Edit:

So that the comments make sense, I’d like to note that I originally proposed using the built-in compile() function. However, @Jian found a simple case that it didn’t handle well. Perhaps it could be adapted, as suggested by @DSM, but then the solution becomes a little less tidy. Maybe not unreasonably so, but if the ast parse-and-dump works as well or better, it’s the more straightforward way.