Question

I have the sequence of tokens like this:

TokenName(alphanum) value(printable) value(printable, optional) value(printable, optional) Literal(';')

by example:

Token1 "stringValue with escaped data eg: \" in it";
Token2 12;
Token3 0xaaaa; // and hex string indicated by 0x
Token4 "value1" 2 0xbbcc;

Result I except is:

[
    ['Token1', 'stringValue with escaped data eg: " in it'],
    ['Token2', 12],
    ['Token3', '0xaaaa'],
    ['Token4', ['value1', 2, '0xbbcc']],
]

Now I have something like this:

import pyparsing as p
non_semi = ''.join(c for c in p.printables if c != ';')
semi = p.Literal(';').suppress()

single_value = p.Word(non_semi) + semi
multi_value = p.Group(
    p.Word(non_semi) +
    p.Word(non_semi) +
    p.Optional(p.Word(non_semi)) +
    semi
)

value = single_value | multi_value
assignment = p.Group(p.Word(p.alphanum) + value)

bnf = p.Group(p.OneOrMore(assignment))
bnf.ignore(p.cStyleComment)

But this code, does not change 12 into int(12) and also gets " chars in strings, can pyparsing handle this value transformations?

EDIT

Ok, I know how to deal with string: QuotedString helps a lot

Answer 1

OK, I have found my answers:

For handling of strings: QuotedString

For handling of int and other conversions, there is a setParseAction method of a token def, witch can take eg. lambda s, l, t: int(s) param witch will do the trick