Question

Im trying to create a parser which sould transalte english sentences into drawn shapes on a canvas. For example:
“Create a red box” should create a box on the canvas which is red.

I came up with this grammar file from a tutorial at your wiki. Dont know if it is correct, would be nice if someone could check it =)

grammar Shrdlu;

tokens {

    //operational tokens
    MOVE        = 'move';
    TRANSFORM   = 'transform';
    CREATE      = 'create';
    MAKE        = 'make';
    ADD         = 'add';
    REMOVE      = 'remove';
    DELETE      = 'delete';

    //shape tokens
    BOX         = 'box';
    RECTANGLE   = 'rectangel';
    CIRCLE      = 'circle';
    TRIANGLE    = 'triangle';
    SHAPE       = 'shape';
    SQUARE      = 'square';

    //color tokens
    RED         = 'red';
    BLUE        = 'blue';
    GREEN       = 'green';
    BLACK       = 'black';
    PURPLE      = 'purple';
    YELLOW      = 'yellow';
    ORANGE      = 'orange';
    PINK        = 'pink';

    //size tokens
    BIG         = 'big';
    LARGE       = 'large';
    TALL        = 'tall';
    SMALL       = 'small';
    TINY        = 'tiny';
    SHORT       = 'short';

    //relation size
    BIGGEST     = 'biggest';
    LARGEST     = 'largest';
    TALLEST     = 'tallest';
    SMALLEST    = 'smallest';   
    SHORTEST    = 'shortest';

    //argument size
    BIGGER      = 'bigger';
    SMALLER     = 'smaller';
    SHORTER     = 'shorter';
    TALLER      = 'taller';
    LARGER      = 'larger';

    //alignment tokens
    LEFT        = 'left';
    RIGHT       = 'right';
    OVER        = 'over';
    UNDER       = 'under';
    ABOVE       = 'above';
    BELOW       = 'below';
    TOP         = 'top';
    BOTTOM      = 'bottom';

    //prefix tokens
    A           = 'a';
    AN          = 'an';
    ALL         = 'all';
    ANY         = 'any';
    EACH        = 'each';
    THE         = 'the';

}

/*------------------------------------------------------------------
 * PARSER RULES
 *------------------------------------------------------------------*/

command
    :   sentence EOF
    |
    ;

sentence
    :   WS? action WS object (WS argument)? WS?
    ;

action

    :   MOVE
    |   TRANSFORM
    |   CREATE
    |   MAKE
    |   ADD
    |   REMOVE
    |   DELETE
    ;

object
    :   prefix WS (property WS)?  shape (WS relation WS object)?
    ;

a    rgument
    :   color
    |   sizearg
    |   alignment
    ;

prefix
    :   A
    |   AN
    |   ALL
    |   ANY
    |   EACH
    |   THE
    ;

shape
    :   BOX
    |   RECTANGLE
    |   CIRCLE
    |   TRIANGLE
    |   SHAPE
    |   SQUARE
    ;

property
    :   size (WS property)?
    |   color (WS property)?
    ;


size
    :   BIG
    |   LARGE
    |   TALL
    |   SMALL
    |   TINY
    |   SHORT
    ;

sizearg
    :   BIGGER
    |   LARGER
    |   SMALLER
    |   TALLER
    |   SHORTER 
    ;

relsize
    :   BIGGEST
    |   SMALLEST
    |   TALLEST
    |   LARGEST
    ;   


relation
    :   alignment
    |   relsize
    ;

color
    :   RED
    |   BLUE
    |   GREEN
    |   BLACK
    |   PURPLE
    |   YELLOW
    |   ORANGE
    |   PINK
    ;

alignment
    :   LEFT
    |   RIGHT
    |   OVER
    |   UNDER
    |   ABOVE
    |   BELOW
    |   TOP
    |   BOTTOM
    ;

/*------------------------------------------------------------------
 * LEXER RULES
 *------------------------------------------------------------------*/

NEWLINE
    :   '\r'? '\n'
    ;

WS
    :   (' '|'\t'|'\n'|'\r')+ {skip();}
    ;

Then i used this code to generate the lexer and parser. What is my next step. How do i use the parser to, for example transalte “create” into the creation of an object. Could someone point me in the right direction?

Answer 1

Because of {skip();} in the WS rule, your lexer will never create any WS tokens. So parser rules that have WS in them, like the sentence rule, will never match. So in that sense, you grammar is wrong.

Richard wrote:

into the creation of an object.

EDIT

You can give your parser custom attributes, like a Graphics object you’re going to paint on (see the @parser::members { ... } section of the grammar below). And embed custom code inside your grammar rules between { and }. As a demo I only added some System.out.println‘s, but you should replace those with your actual painting, of course. I also removed the WS tokens from your parser rules (also note that Create does not equal create!).

The modified grammar:

grammar Shrdlu;

tokens {

    //operational tokens
    MOVE        = 'move';
    TRANSFORM   = 'transform';
    CREATE      = 'create';
    MAKE        = 'make';
    ADD         = 'add';
    REMOVE      = 'remove';
    DELETE      = 'delete';

    //shape tokens
    BOX         = 'box';
    RECTANGLE   = 'rectangel';
    CIRCLE      = 'circle';
    TRIANGLE    = 'triangle';
    SHAPE       = 'shape';
    SQUARE      = 'square';

    //color tokens
    RED         = 'red';
    BLUE        = 'blue';
    GREEN       = 'green';
    BLACK       = 'black';
    PURPLE      = 'purple';
    YELLOW      = 'yellow';
    ORANGE      = 'orange';
    PINK        = 'pink';

    //size tokens
    BIG         = 'big';
    LARGE       = 'large';
    TALL        = 'tall';
    SMALL       = 'small';
    TINY        = 'tiny';
    SHORT       = 'short';

    //relation size
    BIGGEST     = 'biggest';
    LARGEST     = 'largest';
    TALLEST     = 'tallest';
    SMALLEST    = 'smallest';   
    SHORTEST    = 'shortest';

    //argument size
    BIGGER      = 'bigger';
    SMALLER     = 'smaller';
    SHORTER     = 'shorter';
    TALLER      = 'taller';
    LARGER      = 'larger';

    //alignment tokens
    LEFT        = 'left';
    RIGHT       = 'right';
    OVER        = 'over';
    UNDER       = 'under';
    ABOVE       = 'above';
    BELOW       = 'below';
    TOP         = 'top';
    BOTTOM      = 'bottom';

    //prefix tokens
    A           = 'a';
    AN          = 'an';
    ALL         = 'all';
    ANY         = 'any';
    EACH        = 'each';
    THE         = 'the';

}

@parser::members {

  private java.awt.Graphics graphics;

  public ShrdluParser(TokenStream tokens, java.awt.Graphics g) {
    super(tokens);
    graphics = g;
  }
}

/*------------------------------------------------------------------
 * PARSER RULES
 *------------------------------------------------------------------*/

command
    :   sentence EOF
    |
    ;

sentence
    :   action object argument?
    ;

action

    :   MOVE
    |   TRANSFORM
    |   CREATE    {System.out.println("I should create a ...");}
    |   MAKE
    |   ADD
    |   REMOVE
    |   DELETE
    ;

object
    :   prefix property?  shape (relation object)?
    ;

argument
    :   color
    |   sizearg
    |   alignment
    ;

prefix
    :   A
    |   AN
    |   ALL
    |   ANY
    |   EACH
    |   THE
    ;

shape
    :   BOX       {System.out.println("box ...");}
    |   RECTANGLE
    |   CIRCLE
    |   TRIANGLE
    |   SHAPE
    |   SQUARE
    ;

property
    :   size property?
    |   color property?
    ;


size
    :   BIG
    |   LARGE
    |   TALL
    |   SMALL
    |   TINY
    |   SHORT
    ;

sizearg
    :   BIGGER
    |   LARGER
    |   SMALLER
    |   TALLER
    |   SHORTER 
    ;

relsize
    :   BIGGEST
    |   SMALLEST
    |   TALLEST
    |   LARGEST
    ;   


relation
    :   alignment
    |   relsize
    ;

color
    :   RED   {System.out.println("red ...");}
    |   BLUE
    |   GREEN
    |   BLACK
    |   PURPLE
    |   YELLOW
    |   ORANGE
    |   PINK
    ;

alignment
    :   LEFT
    |   RIGHT
    |   OVER
    |   UNDER
    |   ABOVE
    |   BELOW
    |   TOP
    |   BOTTOM
    ;

NEWLINE
    :   '\r'? '\n'
    ;

WS
    :   (' '|'\t'|'\n'|'\r')+ {skip();}
    ;

which can be tested with the class:

import org.antlr.runtime.*;

public class Main {
    public static void main(String[] args) throws Exception {
        ANTLRStringStream in = new ANTLRStringStream("create a red box");
        ShrdluLexer lexer = new ShrdluLexer(in);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        java.awt.Graphics g = null;
        ShrdluParser parser = new ShrdluParser(tokens, g);
        parser.command();
    }
}

Create the lexer and parser from your grammar:

java -cp antlr-3.3.jar org.antlr.Tool Shrdlu.g

then compile all .java source files:

javac -cp antlr-3.3.jar *.java

and run the Main class:

# *nix
java -cp .:antlr-3.3.jar Main

# Windows
java -cp .;antlr-3.3.jar Main

which produces the following output:

I should create a ...
red ...
box ...

And one final thing, you can probably remove the NEWLINE rule: your WS rule catches such chars (and ignores them).