Question

I am working on a simple DSL to transform data extracted from MongoDB. I am using python and pyparsing and have gotten reasonably far in creating a grammar that works for basic operators like +/-*, starting from the examples provided. I am currently stuck on how to get my program to evaluate functions of the form Rank[dbRef]. I can evaluate and do arithmetic on dbRefs through the simple operators, but something is not working with my recursion in evaluating functions. I cannot figure out how to access the dbRef argument that was passed in the function call.

Here is the grammar and associated setParseActions:

# Define parser, accounting for the fact that some fields contain whitespace
chars = Word(alphanums + "_-/")
expr = Forward()
integer = Word(nums).setParseAction(EvalConstant)
real = Combine(Word(nums) + "." + Word(nums)).setParseAction(EvalConstant)

# Handle database field references that are coming out of Mongo
dbRef = Combine(chars + OneOrMore(":") + chars)

dbRef.setParseAction(EvalDBref)

# Handle function calls
functionCall = (Keyword("Rank") | Keyword("ZS") | Keyword("Ntile")) + "[" + dbRef + "]"
functionCall.setParseAction(EvalFunction)
operand = (real | integer) | functionCall | dbRef 

signop = oneOf('+ -')
multop = oneOf('* /')
plusop = oneOf('+ -')

# Use parse actions to attach Eval constructors to sub-expressions

expr << operatorPrecedence(operand,
    [
     (signop, 1, opAssoc.RIGHT, EvalSignOp),
     (multop, 2, opAssoc.LEFT, EvalMultOp),
     (plusop, 2, opAssoc.LEFT, EvalAddOp),
    ])

formulas = ['Rank[Person:Height]']

for f in formulas:
    ret = expr.parseString(f)[0]
    print p + ": " + line + " --> " + str(ret.eval())

Here is the relevant code for my evaluation class. The class DOES get called by the parser, but how do I access the argument that is passed to the function?

# Executes functions contained in expressions
class EvalFunction(object): 
    def __init__(self, tokens): 
        self.value = tokens[0]
    def eval(self):
        func = self.value
        if func == 'Rank':
            # How to evaluate the token that is arg of Function?
            return 'Rank Found';

I think I just need a nudge in the right direction to get to the next stage ..

Answer 1

I sorted this out and wanted to provide an answer. My function evaluation class looks like this:

# Executes functions contained in expressions
class EvalFunction(object):
  pop_ = {}
  def __init__(self, tokens):
    self.func_ = tokens.funcname
    self.field_ = tokens.arg
  def eval(self):
      # Get the name of the requested field and source db
      # Functions can only be called on dbRef so this always done
      v = self.field_.value
      fieldRef = v.split(':')
      source = fieldRef[0]
      field = fieldRef[1]

      # Evaluate the dbRef (get the value from the db)
      val = self.field_.eval()

      if self.func_ == 'Avg':
        rec = db['Stats'].find_one({'_id' : field})   
        return rec['value']['avg']
      elif self.func_ == 'Root':
          return math.sqrt(val)

my function grammar is:

functionCall = funcNames("funcname") + "[" + dbRef("arg") + "]"
functionCall.setParseAction(EvalFunction)