Question

I was writing a simple grammar for a small language when I came across an issue. It’s throwing a NullPointerException when I try to interpret it. Oh, and this is ANTLRWorks 3.

Also – I specified that the language should be Python, but it still does stuff in Java 🙁 Why?

Here’s the input:

program Test1 = 
const t1 : int := 1;
const t2 : int := 2;
var x, y, z : int;
begin
x := 41;
y := 10;
z := 2 4 *;
end Test1.

Here’s my code:

grammar lang;

options {
    language=Python;
    output=AST;
    ASTLabelType=CommonTree;
}

tokens {DECL;} // an imaginary node

start : decl ;

decl : type ID ';' -> ^(DECL type ID)
     ;
type : INTTYPE  // automatic tree construction builds a node for this rule
     | FLOATTYPE
     ;
program
    : 'program' ID '='
    (const | variable)*
    'begin'
    statement*
    'end' ID '.'
    ;
const
    : 'const' ID ':' type ':=' expr ';'
    ;
variable
    : 'var' ID (',' ID)* ':' type ';'
    ;
statement
    : assignment
    ;
assignment
    : ID ':=' expr ';'
    ;



// expressions....fun!

term
    : ID
    | expr
    | INTTYPE
    ;
negation 
    : 'not'* term
    ;
unary
    : ('+' | '-')* negation
    ;
mult
    : unary (unary ('*' | '/' | 'mod'))*
    ;
add
    : mult (mult ('+' | '-'))*
    ;
relation
    : add (add ('==' | '!=' | '<' | '<=' | '>=' | '>'))*
    ;

expr
    : relation (relation ('and' | 'or'))* 
    ;

INTTYPE : 'int' ;
FLOATTYPE : 'float' ;
ID : ('a'..'z' | 'A'..'Z') ('a'..'z' | 'A'..'Z' | '0'..'9')* ;
INT : '0'..'9'+ ;
WS : (' '|'\n' | '\t') {$channel=HIDDEN;} ;

What am I doing wrong?

Answer 1

As Kay already mentioned: there is no need to account for operator precedence in post-fix expressions. Post-fix expressions are just a list of one or more operands and operators. Here’s how to get your grammar working:

grammar lang;

options {
  language=Python;
  output=AST;
}

tokens {
  PROGRAM;
  STATS;
  DECL;
  ASSIGN;
  EXPR;
}

program
  :  'program' id=ID '=' decl* 'begin' statement* 'end' ID '.'
     -> ^(PROGRAM $id ^(DECL decl*) ^(STATS statement*))
  ;

decl
  :  const 
  |  variable
  ;

type 
  :  INTTYPE 
  |  FLOATTYPE
  ;

const
  :  'const' ID ':' type ':=' expr ';' -> ^('const' type ID expr)
  ;

variable
  :  'var' ID (',' ID)* ':' type ';' -> ^('var' type ID+)
  ;

statement
  :  assignment
  ;

assignment
  :  ID ':=' expr ';' -> ^(ASSIGN ID expr)
  ;

expr
  :  exprAtom+ -> ^(EXPR exprAtom+)
  ;

exprAtom
  :  operand 
  |  operator
  ;

operand
  :  INT
  |  ID
  ;

operator
  :  'and' | 'or' | '==' | '!=' | '<' | '<=' | '>=' | '>' | '+' | '-' | '*' | '/' | 'mod' | 'not'
  ;

INTTYPE   : 'int' ;
FLOATTYPE : 'float' ;
ID        : ('a'..'z' | 'A'..'Z') ('a'..'z' | 'A'..'Z' | '0'..'9')* ;
INT       : '0'..'9'+ ;
WS        : (' '|'\n' | '\t') {$channel=HIDDEN;} ;

Now generate a lexer and parser (Python source files!) from it by executing the following on the command line:

java -cp antlr-3.1.3.jar org.antlr.Tool lang.g

And if you now execute the following script

#!/usr/bin/env python
import antlr3
from antlr3 import *
from antlr3.tree import *
from langLexer import *
from langParser import *

def print_level_order(tree, indent):
  print '{0}{1}'.format('   '*indent, tree.text)
  for child in tree.getChildren():
    print_level_order(child, indent+1)

input = """
program Test1 = 
const t1 : int := 1;
const t2 : int := 2;
var x, y, z : int;
begin
x := 41;
y := 10;
z := 2 4 *;
end Test1.
"""
char_stream = antlr3.ANTLRStringStream(input)
lexer = langLexer(char_stream)
tokens = antlr3.CommonTokenStream(lexer)
parser = langParser(tokens)
tree = parser.program().tree 
print_level_order(tree, 0)

you’ll see the following being printed to the console:

PROGRAM
   Test1
   DECL
      const
         int
         t1
         EXPR
            1
      const
         int
         t2
         EXPR
            2
      var
         int
         x
         y
         z
   STATS
      ASSIGN
         x
         EXPR
            41
      ASSIGN
         y
         EXPR
            10
      ASSIGN
         z
         EXPR
            2
            4
            *