Question

I’m working on a little antlr problem. In my small custom DSL I want to be able to do a compare action between fields. I’ve got three fieldtypes (String, Int, Identifier) the Identifier is a variable name. I made a big specification but i’ve reduced my problem to a smaller grammer.

The problem is that when I try to use the String grammar notation, which you can add to your grammer using antlrworks, my Strings are seen as an identifier. This is my grammar:

grammar test;

x
    : 'FROM' field_value EOF
    ;

field_value
    : STRING
    | INT
    | identifier
    ;

identifier
    : ID (('.' '`' ID '`')|('.' ID))?
    | '`' ID '`' (('.' '`' ID '`')|('.' ID))?
    ;

ID  :   ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
    ;

INT :   '0'..'9'+
    ;

STRING
    :  '"' ( ESC_SEQ | ~('\\'|'"') )* '"'
    ;

fragment
HEX_DIGIT : ('0'..'9'|'a'..'f'|'A'..'F') ;

fragment
ESC_SEQ
    :   '\\' ('b'|'t'|'n'|'f'|'r'|'\"'|'\''|'\\')
    |   UNICODE_ESC
    |   OCTAL_ESC
    ;

fragment
OCTAL_ESC
    :   '\\' ('0'..'3') ('0'..'7') ('0'..'7')
    |   '\\' ('0'..'7') ('0'..'7')
    |   '\\' ('0'..'7')
    ;

fragment
UNICODE_ESC
    :   '\\' 'u' HEX_DIGIT HEX_DIGIT HEX_DIGIT HEX_DIGIT
    ;

When I try to parse the following string FROM "Hello!" it returns a parsetree like this

       <grammar test>
             |
             x
             |
----------------------------
 |           |             |
FROM    field_value        !
             |
         identifier
             |
          "Hello

It parses what I think should be a string to an identifier thought my identifier doesn’t say anything about double quoets so it shouldn’t match.

Besides I think my definition for a string is wrong, even though antlrworks generated it for me. Does anybody know why this happens?

Cheers!

Answer 1

There’s nothing wrong with your grammar. The things that is messing it up for you is most probably the fact that you’re using ANTLRWorks’ interpreter. Don’t. The interpreter doesn’t work well.

Use ANTLRWorks’ debugger instead (in your grammar, press CTRL + D), which works like a charm. This is what the debugger shows after parsing FROM "Hello!":