Question

I have a rule that I would like to split as I would like to re-use the same sub rule in other rules too. The original version works fine:

type
  : ( basicType -> basicType )
  ( '*' -> ^(TYPE_POINTER basicType)
  | '[' ']' -> ^(TYPE_DYN_ARRAY basicType)
  | '[' IntegerLiteral ']' -> ^(TYPE_STATIC_ARRAY IntegerLiteral basicType)
  | '[' IntegerLiteral '..' IntegerLiteral ']' -> ^(REF_TYPE_SLICE IntegerLiteral IntegerLiteral basicType)
  | '[' type ']' -> ^(TYPE_MAP_ARRAY type basicType)
  )?
  ;

This rule makes it possible to take a normal type (basicType, which can be a reference or a primary type, like int, char, etc..) and place under a different AST node, which makes it pointer, array, or so. This is very common thing in C like languages.

However, if I split like this:

basicType2
  : '*' -> ^(TYPE_POINTER)
  | '[' ']' -> ^(TYPE_DYN_ARRAY)
  | '[' IntegerLiteral ']' -> ^(TYPE_STATIC_ARRAY IntegerLiteral)
  | '[' IntegerLiteral '..' IntegerLiteral ']' -> ^(REF_TYPE_SLICE IntegerLiteral IntegerLiteral)
  | '[' type ']' -> ^(TYPE_MAP_ARRAY type)
  ;

type
  : ( basicType -> basicType )
    ( basicType2 -> ^(basicType2 basicType) )?
  ;

Everything looks great and the parsing will not be effected, however in the AST the basicType2’s children are missing completely. In the case of TYPE_STATIC_ARRAY, or REF_TYPE_SLICE, or TYPE_MAP_ARRAY, only the root will be copied above the type but sub nodes are missing.

I debugged the parser code, and it seems to me that the CommonTree class’ copy constructor which is called when the doesn’t copy the children, only the token and the source range information. Is there any way to bring a CommonTree node along with it’s children to the top of an other with rewrite rules?

Answer 1

Always use a single token as the root of a tree: not another tree, as you’re doing with basicType2 in:

type
  : ( basicType -> basicType )
    ( basicType2 -> ^(basicType2 basicType) )?
  ;

Try this instead:

grammar ...

options { ... }

tokens { T2; }

...

type
  : ( basicType -> basicType )
    ( basicType2 -> ^(T2 basicType2 basicType) )?
  ;

...