Question

Is there a known way to generate an “Expected token” list when a syntax error happens ? I’m using Lemon as parser generator.

Answer 1

This seems to work:

%syntax_error { 
        int n = sizeof(yyTokenName) / sizeof(yyTokenName[0]);
        for (int i = 0; i < n; ++i) {
                int a = yy_find_shift_action(yypParser, (YYCODETYPE)i);
                if (a < YYNSTATE + YYNRULE) {
                        printf("possible token: %s\n", yyTokenName[i]);
                }
        }
}

It tries all possible tokens and prints those that are applicable in the current parser state.

Note that when an incorrect token comes, the parser doesn’t immediately call syntax_error, but it tries to reduce what’s on stack hoping the token can be shifted afterwards. Only when nothing else can be reduced and the current token cannot be shifted, the parser calls syntax_error. The reductions will change parser state, which means that you may see less tokens than would have been applicable before the reductions. It should be sufficient for error reporting though.