Question

I am trying to build a parser with Bison/Yacc to be able to parse a flow of token done by another module. The tokens are already listed in a enumeration type as follow:

// C++ header file
enum token_id {
  TokenType1         = 0x10000000,
  TokenType2         = 0x11000000,
  TokenType3         = 0x11100000,
  //... and the list go on with about 200/300 line
};

I have gone through the documentation of bison many times but I couldn’t find a better solution than copying each token in the Bison file like this:

/* Bison/Yacc file */
%token TokenType1 0x10000000
%token TokenType2 0x11000000
%token TokenType3 0x11100000
//...

If I have to do it like that, It will become pretty hard to maintain the file if the other module specification change (which happen quite oftenly).

Could you please tell me how to do it, or point me in the good direction (any idea/comment is welcome). It would greatly help me! Thanks in advance.

Answer 1

Instead of doing :

/* Bison/Yacc file */
%token TokenType1 0x10000000
%token TokenType2 0x11000000
%token TokenType3 0x11100000
//...

You just need to include the file with the token type in the declaration part

#include "mytoken_enum.h"
// ...
%token TokenType1
%token TokenType2
%token TokenType3 
//...

EDIT: This can not be done:

As you see from the numbers above,
Bison just numbers the tokens
sequentially, and it is used shifted
in parser lookup tables as indices,
for speed simply. So Bison does not
support that, I feel sure, and it
would not be easy to fit with the
implementation model.

Just need to wrapper to convert the real token to yacc/bison token (eg: via yylex())