Question

I’m trying to parse a simple subset of SQL using antlr4.

My grammar looks like this:

grammar Query;
query : select;
select : 'select' colname (','  colname)* 'from' tablename;
colname : COLNAME;
tablename : TABLENAME;
COLNAME: [a-z]+ ;
TABLENAME : [a-z]+;
WS : [ \t\n\r]+ -> skip ; // skip spaces, tabs, newlines

I am testing this with a simple java application as follows:

import java.io.ByteArrayInputStream;
import java.io.InputStream;

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;

public class Test {
    public static void main(String[] args) throws Exception {
        // create a CharStream that reads from standard input

        InputStream is = new ByteArrayInputStream("select one,two ,three from table".getBytes());

        ANTLRInputStream input = new ANTLRInputStream(is);

        // create a lexer that feeds off of input CharStream
        QueryLexer lexer = new QueryLexer(input);


        // create a buffer of tokens pulled from the lexer
        CommonTokenStream tokens = new CommonTokenStream(lexer);

        // create a parser that feeds off the tokens buffer
        QueryParser parser = new QueryParser(tokens);

        ParseTree tree = parser.query(); // begin parsing at init rule

        System.out.println(tree.toStringTree(parser)); // print LISP-style tree

    }
}

The output I get is as follows:

line 1:27 mismatched input 'table' expecting TABLENAME
(query (select select (colname one) , (colname two) , (colname three) from (tablename table)))

What I don’t understand is why the parser seems to be picking up “table” as the tablename in the parser tree, but yet I’m also getting an error thrown. What am I missing?

Thanks

Andrew

Answer 1

You cannot have two lexer rules that match the same (at least, not in the same mode/state…):

...
COLNAME: [a-z]+ ;
TABLENAME : [a-z]+;
...

Do this instead:

grammar Query;
query     : select;
select    : 'select' colname (',' colname)* 'from' tablename;
colname   : ID;
tablename : ID;
ID        : [a-z]+;
WS        : [ \t\n\r]+ -> skip;