Question

We have an older C++ tool that generates some python code automatically. I tried to slog through the C++ source tool, today and pretty much wanted to shoot my self. The thing is what i want to do, is clean up the source created by the tool and link the classes to our internal documentation system via adding sphinx tags.

Now what i am wondering is there some sort of wonderful tool for parsing python code within python?

There is alot of stuff like this:

foo._methods_.append()

Snip 500 lines

foo._methods_.append()

ANy suggestions?

Basically i have a functional but insanely messy code structure, i basically want to extract certain chunks, move them to their own files. And cleanup all the miscellanous things that get created.

I looked quickyl at both parser and AST but i cannot find any real examples of it being used.

Answer 1

You may tokenize python code to parse individual tokens using tokenize module. e.g. Script to remove Python comments/docstrings

or you can use the parser module

or use ast module