Question

Given a String like..

(a+(a+b)), (d*e) :- (e-f)

Note: (d*e) and (e-f) are different expressions. How can I fetch the expressions from this string. I have the grammar defined as..

parse returns [String value]
  :  addExp {$value=$addExp.value;} EOF
  ;

addExp returns [String value]
  :  multExp {$value=$multExp.value;} (('+' | '-' | '*') multExp{$value+= '+' + $multExp.value;})*
  ;

multExp returns [String value]
  :  atom {$value=$atom.value;} (('*' | '/') atom {$value+=$atom.value;)*
  ;

atom returns [String value]
  :  x=ID {$value=$x.text;}
  |  '(' addExp ')' {$value='('+$addExp.value+')';}
  ;

ID    : 'a'..'z' | 'A'..'Z';

I tried..

ANTLRStringStream a=new ANTLRStringStream("(a+(a+b)), (d*e) :- (e-f)");
SLexer l=new SLexer(a);
CommonTokenStream c=new CommonTokenStream(l);
SParser p=new Sparser(c);

String exp;
while(exp = p.parse())
{
 System.out.println(exp);
}

I’m thinking of something like hasNext() and then fetching.

Answer 1

Your lexer rules TEXT possibly matches an empty string, causing the lexer to create an infinite amount of tokens. Also, you don’t need all those return statements after your rule: you can simply grab what a parser (or lexer) rule matched by adding .text after it.

You could let your parser return a List<String>, or let it return a single String repeatedly invoke that parser rule until EOF is encountered.

A little demo:

grammar T;

@parser::members {
  public static void main(String[] args) throws Exception {
    String src = "likes(a, b) :- likes(a, X), likes(X, b). hates(a, b) " + 
        ":- hates(a,X), hates(X,b). likes(a,b) :- says(god, likes(a,b)).";
    TLexer lexer = new TLexer(new ANTLRStringStream(src));
    TParser parser = new TParser(new CommonTokenStream(lexer));
    List<String> statements = parser.parse();
    for(String s : statements) {
      System.out.println(s);
    }
  }
}

parse returns [List<String> statements]
@init{$statements = new ArrayList<String>();}
  :  (statement {$statements.add($statement.text);} ~TEXT+)+ EOF
  ;

statement
  :  TEXT OPAR params CPAR
  ;

params
  :  (param (COMMA param)*)?
  ;

param
  :  TEXT
  |  statement
  ;

COMMA : ',';
OPAR  : '(';
CPAR  : ')';
TEXT  : ('a'..'z' | 'A'..'Z')+;
SPACE : (' ' | '\t') {$channel=HIDDEN;};
OTHER : . ;

Note that ~TEXT+ in the parse rule matches one or more tokens other than TEXT.

If you now create a lexer and parser and run the TParser class:

*nix/MacOS

java -cp antlr-3.3.jar org.antlr.Tool T.g
javac -cp antlr-3.3.jar *.java
java -cp .:antlr-3.3.jar TParser

or

Windows

java -cp antlr-3.3.jar org.antlr.Tool T.g
javac -cp antlr-3.3.jar *.java
java -cp .;antlr-3.3.jar TParser

you will see the following being printed to your console:

likes(a, b)
likes(a, X)
likes(X, b)
hates(a, b)
hates(a,X)
hates(X,b)
likes(a,b)
says(god, likes(a,b))

---

EDIT

And here’s how to return a single String opposed to a List<String>:

@parser::members {
  public static void main(String[] args) throws Exception {
    String src = "likes(a, b) :- likes(a, X), likes(X, b). hates(a, b) " + 
        ":- hates(a,X), hates(X,b). likes(a,b) :- says(god, likes(a,b)).";
    TLexer lexer = new TLexer(new ANTLRStringStream(src));
    TParser parser = new TParser(new CommonTokenStream(lexer));
    String s;
    while((s = parser.parse()) != null) {
      System.out.println(s);
    }
  }
}

parse returns [String s]
  :  statement ~(TEXT| EOF)* {$s = $statement.text;}
  |  EOF                     {$s = null;}
  ;