Question

I’m trying to build a simple grammar to parse a .Net type name string, supporting generics. I admit to being completely new to building grammars in any language. A type string might look like the following.

Foo.Bar.Blah(Mom.Dad, Son.Daughter(Frank.Bob), Dog)

Basically, it’s recursive. Ya’ll should understand this.

I’m completely out in the woods with this one. Not sure how to begin. What I’ve built currently, which doesn’t actually work, is this:

tree grammar XmlTypeName;

options {
    language=CSharp2;
}

RPAREN
  : '('
  ;

LPAREN
  : ')'
  ;

SEP
  : ','
  ;

TYPE  
  : ('a'..'z'|'A'..'Z'|'0'..'9'|'_')+
  ;

prog
  : type;

type
  : TYPE (RPAREN type (SEP type)? LPAREN)? (EOF)?
  ;

This doesn’t even get close to working. Antlr3.exe throws errors saying that RPARAM and LPARAM aren’t allowed in a tree parser. Is a tree parser even what I need?

I’d like to produce a simple AST that lets me navigate down the types.

Answer 1

No, you shouldn’t use a tree grammar. A tree grammar is used after a parser has created an AST. Simply remove the keyword tree from it.

A couple of other remarks:

• you want to match one or more comma separated types inside parenthesis, but you used type (SEP type)?, which matches one or two types. You’ll need type (SEP type)* instead;
• you didn’t account for the . inside the types;
• you should discard literal spaces in the lexer.

Something like this will do the trick, most probably:

grammar XmlTypeName;

options {
  language=CSharp2;
}

prog
 : type EOF
 ;

type
 : name (RPAREN type (SEP type)* LPAREN)? 
 ;

name
 : ID (DOT ID)*
 ;

RPAREN
 : '('
 ;

LPAREN
 : ')'
 ;

SEP
 : ','
 ;

DOT
 : '.'
 ;

ID  
 : ('a'..'z'|'A'..'Z'|'0'..'9'|'_')+
 ;

SPACE
 : (' '|'\t')+ {Skip();} // if 'Skip()' doesn't work, try 'skip()'
 ;

However, the above just creates a flat list of tokens. If you want to create a proper AST, you need to “tell” ANTLR which nodes/tokens are root tokens, and which ones to discard (like the comma’s, parenthesis, …).

grammar XmlTypeName;

options {
  output=AST;
  language=CSharp2;
}

tokens {
  TYPE;
  NAME;
}

prog
 : type EOF -> type
 ;

type
 : name (RPAREN type (SEP type)* LPAREN)? -> ^(TYPE name type*)
 ;

name
 : ID (DOT ID)* -> ^(NAME ID+)
 ;

RPAREN
 : '('
 ;

LPAREN
 : ')'
 ;

SEP
 : ','
 ;

DOT
 : '.'
 ;

ID  
 : ('a'..'z'|'A'..'Z'|'0'..'9'|'_')+
 ;

SPACE
 : (' '|'\t')+ {skip();}
 ;

which creates the following AST:

More info about creating AST’s with ANTLR: How to output the AST built using ANTLR?