Question

What does the following mean:

… and StringTokenizer respects exactly five whitespace
characters and nothing else.

http://code.google.com/p/guava-libraries/wiki/StringsExplained#Splitter

Answer 1

Presumably it means that StringTokenizer will break on \n, \r, space, TAB, and formfeed, by default. From the source for the simplest constructor:

this(str, " \t\n\r\f", false);

You see the five whitespace characters listed there. This is not really a big deal, though, because you can specify your own delimiters using one of the other constructors. StringTokenizer has plenty of other problems; the default set of whitespace characters is the least of its worries.