Question

All,

I’m writing a very simplistic parser with python PLY. It mostly does the job, but for many of the lines of input, I get a Syntax error from yacc. Here is the lexer and parser code, slightly modified for easier testing:

tokens = ('VAR', 'NUMBER', 'CLOSE', 'JUNK')

# Tokens

t_VAR     = r'%[mM]\['
t_CLOSE   = r'\]'
t_JUNK    = r'.'

# Ignored characters
t_ignore = " \t\r"

def t_NUMBER(t):
    r'\d+'
    try:
        t.value = int(t.value)
    except ValueError:
        print("Integer value too large %d", t.value)
        t.value = 0
    return t

def t_newline(t):
    r'\n+'
    t.lexer.lineno += t.value.count("\n")

def t_error(t):
    print("Illegal character '%s'" % t.value[0])
    t.lexer.skip(1)

# Build the lexer
import ply.lex as lex
lex.lex()

# Parsing rules

def p_statement(p):
    '''statement : field'''
    try:
        print p[1]
    except IndexError:
        pass

def p_trash(p):
    '''statement : JUNK'''
    pass

def p_field(p):
    '''field : VAR NUMBER CLOSE'''
    #print p[1], p[2], p[3]
    p[0] = p[2]

def p_error(p):
    print("Syntax error at '%s'" % repr(p)) #p.value)

import ply.yacc as yacc
yacc.yacc()

For a sample: yacc.parse('.set %m[702] $substr($currentlength,2,$currentpg)') which gives as output:

Syntax error at 'LexToken(JUNK,'s',1,1)'
Syntax error at 'LexToken(JUNK,'$',1,13)'

It should output 702 only.

Answer 1

Your top level rule requires a single statement. p_trash matches the first ‘.’ and returns a statement and there is no top level rule to allow it to continue. You could do something like:

def p_junk(p):
    '''statement | JUNK statement'''

You could also do something like this (and create a list of statements):

def p_statements(p):
    '''statements | statement statements
                  | empty'''