Question

Is there some way to just get the next token from a file in Python, as for example the Scanner class does in Java?

File file = new File("something");
Scanner myinput = new Scanner(file);
double a = myinput.nextDouble();
String s = myinput.next();

I’d like to ignore whitespaces, tabs, newlines and just get the next int/float/word from the file. I know I could read the lines and build something like Scanner myself, but I’d like to know if there isn’t already something that I could use.

I’ve searched around but could only find line-oriented methods.

Thank you!

Answer 1

Check out the shlex-module in the standard library: http://docs.python.org/library/shlex.html

import shlex
import StringIO # use in place of files

list(shlex.shlex(StringIO.StringIO('Some tokens. 123, 45.67 "A string with whitespace"')))

It does not handle floats the way you seem to want. Maybe you can extend or modify it.