Question

I am trying to find all occurrences of a literal float value in Python code. Can I do that in Komodo (or in any other way)?

In other words, I want to find every line where something like 0.0 or 1.5 or 1e5 is used, assuming it is interpreted by Python as a float literal (so no comments, for example).

I’m using Komodo 6.0 with Python 3.1.

If possible, a way to find string and integer literals would be nice to have as well.

Answer 1

Our SD Source Code Search Engine (SCSE) can easily do this.

SCSE is a tool for searching large source code bases, much faster than grep, by indexing the elements of the source code languages of interest. Queries can then be posed, which use the index to enable fast location of search hits. Queries and hits are displayed in a GUI, and a click on a hit will show the block of source code containing the hit.

The SCSE knows the lexical structure of each language it has indexed with the precision as that langauge’s compiler. (It uses front ends from family of accurate programming language processors; this family is pretty large and happens to include the OP’s target langauge of Python/Perl/Java/…). Thus it knows exactly where identifiers, comments, and literals (integral, float, character or string) are, and exactly their content.

SCSE queries are composed of commands representing sequences of language elements of interest. The query

'for' ... I '=' N=103

finds a for keyword near (“…”) an arbitrary identifier(I) which is initialized (“=”) with the numeric value (“N”) of 103. Because SCSE understands the language structure, it ignores language-whitespace between the tokens, e.g., it can find this regardless off intervening blanks, whitespace, newlines or comments.

The query tokens I, N, F, S, C represent I(dentifier), Natural (number), F(loat), S(tring) and C(omment) respectively. The OP’s original question, of finding all the floats, is thus the nearly trivial query

F

Similarly for finding all String literals (“S”) and integral literals (“N”). If you wanted to find just copies of values near Pi, you’d add low and upper bound constraints:

F>3.14<3.16

(It is pretty funny to run this on large Fortran codes; you see all kinds of bad approximations of Pi).

SCSE won’t find a Float in a comment or a string, because it intimately knows the difference. Writing a grep-style expression to handle all the strange combinations to eliminate whitespace or surrounding quotes and commente delimiters should be obviously a lot more painful. Grep ain’t the way to do this.