Question

I dug through the whole documentation and couldn’t find an example. All the examples either parse character data or use Spirit.Lex. Forgive me if I missed something.

Can someone give an example for, or point to a tutorial on, how to use Boost.Spirit.Qi with my custom lexer? E.g.:

vector<MyTokenType> tokens = GetTokens();
// use boost spirit to work with MyTokenType on per-token granularity

Answer 1

You will have to do sevaral things:

a) expose the token sequence as a range of iterators, which will have to be passed to parse/phrase_parse
b) add a default conversion operator to your token type exposing the token id

struct token
{
    operator int() const { return id; }
};

that allows to use qi::char_(ID) as a parser component matching a token with the token id ID.

Integrating attributes (token values) is more involved, look at Spirit.Lex how it can be done.