Question

This is taken right from The “C++ Programming Language” by Bjarne Stroustrup. I would just like some clarification on how he accumulating the digits into the variable (int number_value). Please don’t rip on the code, I didn’t write it (Complete code from chapter 6 at bottom of post).

Specifically as the parser is calling the lexer, how is the lexer building up a number by using cin. I believe the answer is in these eight lines, but I would like an explanation of how it works.

if( isalpha( ch ) ) {
    (*input).putback( ch );
    (*input) >> string_value;
    return curr_tok=NAME;
} else {
    error( "bad token " );
    return curr_tok=PRINT;
}

It appears to me that the first time get_token( ) is called, it puts the full expression_list into cin or whatever input stream input points to (inside get_token( )).

(*input) >> ch;

I know ch is declared as a char, but what happens if you type 123.4+5.432; (assuming input is cin) cin now contains the “string” 123.4+5.432 contained in its stream. Then we move to the switch statement in the lexer (get_token( )). I am assuming: :

ch == 1?

at this point? Next inside the switch statement, we would “fall through” to the ‘.’ case. Here we place ‘1’ back into the stream and write it out to number_value?

(*input).putback( ch );
(*input) >> number_value;

Now number_value = 1, and we return to the parser. Since we found a NUMBER it calls get_token( ) again. And cin operator<< is called again. Wouldn’t the next call to (*input)>> number_value place 2 into number value overwriting the 1 (assuming that the input is still 123.4+5.432)? What happens here. I guess I need a better understanding of how streams work. If someone could take the time, and give a brief explanation and point me to a good resource I would greatly appreciated.

Thank you,

Matthew Hoggan

For those that don’t have the book, the code is:

#include <iostream>
#include <stdlib.h>
#include <string>
#include <sstream>
#include <map>
#include <cctype>

std::istream *input;

double number_value;
int no_of_errors;
std::string string_value; 
std::map<std::string,double> table;

enum Token_value {
    NAME,        NUMBER,        END,
    PLUS='+',    MINUS='-',     MUL='*',       DIV='/',  
    PRINT=';',   ASSIGN='=',    LP='(',        RP=')'
}; 

Token_value curr_tok=PRINT;

double expr( bool );
double term( bool );
double prim( bool );
Token_value get_token( );
double error( std::string s ) {
    no_of_errors++;
    std::cerr << "error: " << s << std::endl;
    return 1.0;
}

Token_value get_token( ) {
    char ch = 0;
    (*input) >> ch;

    switch( ch ) {
        case 0: {
            return curr_tok=END;
        }
        case ';': 
        case '*':
        case '/':
        case '+':
        case '-':
        case '(':
        case ')':
        case '=': {
            return curr_tok = static_cast<Token_value>( ch );
        }
        case '0':
        case '1':
        case '2':
        case '3':
        case '4':
        case '5':
        case '6':
        case '7':
        case '8':
        case '9':
        case '.': {
            (*input).putback( ch );
            (*input) >> number_value;
            return curr_tok=NUMBER;
        }
        default: {
            if( isalpha( ch ) ) {
                (*input).putback( ch );
                (*input) >> string_value;
                return curr_tok=NAME;
            } else {
                error( "bad token " );
                return curr_tok=PRINT;
            }
        }
    }
}

int main( int argc, char *argv[ ] ) {
    switch( argc ) {
        case 1: {
            input = &std::cin;
            break;
        }
        case 2: {
            input = new std::istringstream( argv[1] );
            break;
        }
        default: {
            error(" To many arguments" );
            return 1;
        }
    }
    table["pi"] = 3.1415926535897932385;
    table["e"] = 2.7182818284590452354;

    while( (*input) ) {
        get_token( );
        if( curr_tok == END ) {
            break;
        }
        if( curr_tok == PRINT ) {
            continue;
        }
        std::cout << expr( false ) << std::endl;
    }

    if( input != &std::cin ) {
        delete input;
    }

    return 0;
}

double expr( bool get ) {
    double left = term( get );

    for( ; ; ) {
        switch( curr_tok ) {
            case PLUS: {
                left += term( true );
                break;
            }
            case MINUS: {
                left -= term( true );
                break;
            }
            default: {
                return left;
            }
        }
    }
}

double term( bool get ) { 
    double left = prim( get );
    for( ; ; ) {
        switch( curr_tok ) {
            case MUL: {
                left *= prim( true );
                break;
            }
            case DIV: {
                if( double d = prim( true ) ) {
                    left /= d;
                    break;                   
                }
                else {
                    return error( "divide by 0" );
                }
            }
            default: {
                return left;
            }
        }
    }
}

double prim( bool get ) {
    if( get ) {
        get_token( );
    }

    switch( curr_tok ) {
        case NUMBER: {
            double v = number_value;
            get_token( );
            return v;
        }
        case NAME: {
             double &v = table[string_value];
             if( get_token( ) == ASSIGN ) {
                 v = expr( true );
                 return v;
             }
        }
        case MINUS: {
            return -prim( true );
        }
        case LP: {
            double e = expr( true );
            if( curr_tok != RP ) {
                return error( "')' expected" );
            }
            get_token( );
            return e;
        }
        default: {
            return error( "primary expected" );
        }
    }
}

Answer 1

The ‘trick’ is caused by the differing behaviour of the following three lines:

char ch;                   std::cin >> ch;
std::string string_value;  std::cin >> string_value;
double number_value;       std::cin >> number_value;

The first just gets a single character, the second and third get multiple characters to build a variable of the correct type.

Strings overload the global operator>> function to provide a version for strings, and this version uses whitespace as the delimiter (if you need to input spaces with your string, you should look into getline).

The double version uses the istream& operator>> (double& val); member function and will read characters only while they make sense in forming a double value.

So, let’s say you enter abc. The code cin >> ch will populate ch with the character 'a', removing it from the input stream. You will then detect this with isapha in the default case since it matches none of the other cases.

At that point, you push that character 'a' back on to the input stream so you can re-read it, and execute cin >> string_value which gets the entire string abc, not a single character.

Similarly, if you entered 3.14159, it would be caught by the case '3' check, the character would be pushed back on to the input stream, and cin >> number_value would then get the whole value.