Question

I’m using Bison and I’ve generated a quite complex grammar. The trouble is that my first test case is failing- but Bison will only say “syntax error”. Is there any way to ask Bison to output the rule that failed to match and the token that is a problem? I used

%define parse.trace true

but still only get syntax error as the output.

Answer 1

None of the Yacc-based parsers does anything much better than ‘syntax error’ when there is a problem; it is largely up to you to improve on that.

There are a couple of things you can do fairly readily.

One is to instrument your lexical analyzer so it prints out the tokens it finds as it returns them to the parser proper. This tells you which token the grammar is failing on, and what tokens were supplied beforehand.

The other is to compile with Yacc debug enabled, and then turn it on. This requires -DYYDEBUG=1 and setting the variable yydebug to a non-zero value (conventionally 1). The first step compiles the extra information into the grammar; the second step enables the output.

From the Bison 2.4.3 manual:

§8.2 Tracing Your Parser

If a Bison grammar compiles properly but doesn’t do what you want when it runs, the
yydebug parser-trace feature can help you figure out why.

There are several means to enable compilation of trace facilities:

• the macro YYDEBUG

  Define the macro YYDEBUG to a nonzero value when you compile the parser.
  This is compliant with POSIX Yacc. You could use -DYYDEBUG=1 as a compiler
  option or you could put #define YYDEBUG 1 in the prologue of the grammar
  file (see Section 3.1.1 [The Prologue], page 47).

• the option -t, --debug

  Use the -t option when you run Bison (see Chapter 9 [Invoking Bison],
  page 117). This is POSIX compliant too.

• the directive %debug

  Add the %debug directive (see Section 3.7.12 [Bison Declaration Summary],
  page 72). This is a Bison extension, which will prove useful when Bison will
  output parsers for languages that don’t use a preprocessor. Unless POSIX and
  Yacc portability matter to you, this is the preferred solution.

We suggest that you always enable the debug option so that debugging is always possible.

The trace facility outputs messages with macro calls of the form YYFPRINTF (stderr,
format, args) where format and args are the usual printf format and variadic arguments.
If you define YYDEBUG to a nonzero value but do not define YYFPRINTF, <stdio.h>
is automatically included and YYFPRINTF is defined to fprintf.

Once you have compiled the program with trace facilities, the way to request a trace is
to store a nonzero value in the variable yydebug. You can do this by making the C code do
it (in main, perhaps), or you can alter the value with a C debugger.

Each step taken by the parser when yydebug is nonzero produces a line or two of trace
information, written on stderr. The trace messages tell you these things:

• Each time the parser calls yylex, what kind of token was read.
• Each time a token is shifted, the depth and complete contents of the state stack (see
  Section 5.5 [Parser States], page 95).
• Each time a rule is reduced, which rule it is, and the complete contents of the state
  stack afterward.