Question

A somewhat vague question, I apologize in advance.

I’m building the tokenizing portion of a small parser with help of the book Building Parsers with Java. It uses PushbackReader and the String contained within as a way to first detect the first character of the given string then sends the PushbackReader to the appropriate state (the state then builds the token as a separate object containing a String).

PushbackReader seems to only be used if no other characters of use are found within the the stream. It then unreads the last character.

Is it possible to do the same thing with a CharBuffer’s append? Preferably something that doesn’t require the buffer to be predefined.

Answer 1

Based on what I see, he chose PushbackReader for two reasons:

1. He needed a reader that could handle individual characters.
2. He needed to backup in the stream because when tokenizing he needed to see one character or more ahead to decide if the current char was part of the token.

For example with the method WhitespaceState.nextToken he is skipping whitespace characters. He pulls off a character and looks at it. If it is a whitespace char he pulls the next char. When he finally pulls a character that is not whitespace, he puts it back into the stream so the next method that looks at the stream will be looking at the correct character.

While you could replace it with something more simple that has just two methods, read(), and unread(), you have to remember that by doing so you will probably be

1. Reading in the entire input, and then processing the input. So if you have a large file you will be eating up memory to store it.
2. Reading the input once as a stream, but storing the char(s) from unread() and passing them around in a separate structure.

With PushbackReader, he is reading and processing through the input once, he does not have to buffer the entire input, nor is he having to store the unread() characters and pass them around separately