Question

I am missing something about how recursion work in Python. I have put in place the following method in place to tokenize a sentence:

def extractIngredientInfo(ingredientLine, sectionTitle):

    print 'extractIngredientInfo' + ingredientLine

    # set-up some default values for variables that will contains the extracted datas
    usmeas             = 'N.A'
    othermeas          = 'N.A'

    p_ingredientalt     = re.compile('\(or\s(.*?)\)')
    malt = p_ingredientalt.search(ingredientLine)
    if malt:
        ingredientAlt = malt.group(1)
        ingredientLine = ingredientLine.replace(malt.group(0), '').strip()
        print 'NEW LINE TO TREAT(ALT)' + ingredientLine
        extractIngredientInfo(ingredientLine, sectionTitle)
        usmeas,othermeas = extractOneIngredientInfo(ingredientAlt)
        print 'MALT'
        ingredient 
        yield usmeas, othermeas
        #return;

    p_ingredientpurpose = re.compile('\(for\s(.*?)\)') 
    mpurpose = p_ingredientpurpose.search(ingredientLine)
    if mpurpose:
        ingredientPurpose = mpurpose.group(1)
        ingredientLine = ingredientLine.replace(mpurpose.group(0), '').strip()
        print 'NEW LINE TO TREAT(FOR)' + ingredientLine
        extractIngredientInfo(ingredientLine, sectionTitle)
        usmeas,othermeas = extractOneIngredientInfo(ingredientPurpose)
        print 'MPURPOSE'
        yield usmeas,othermeas
        #return;

    usmeas,othermeas = extractOneIngredientInfo(ingredientLine)
    print 'FINAL'
    yield usmeas, othermeas

when i am making a calling to this function, I have a match for malt which should lead to an immediate call to the recursive function extractIngredientInfo but this never happening (I don’t see the second call to print 'extractIngredientInfo' + ingredientLine. Is there any specific reason this is not happening?

Answer 1

Your function returns a generator because it uses yield statements. A generator is paused until you request the next value.

This means that a generator function doesn’t do anything until you call .next() it, or use it as an iterator in a loop:

>>> def foo():
...     print 'Foo called'
...     yield 'bar'
...     print 'Still in foo'
...     yield 'baz'
... 
>>> foogen = foo()
>>> foogen.next()
Foo called
'bar'
>>> foogen.next()
Still in foo
'baz'
>>> for val in foo():
...     pass
... 
Foo called
Still in foo

Notice how the message Foo called is not printed until I call .next() on the generator.

You only call your recursive function, but that returns a generator that you then discard. The code itself is never executed because it is kept on hold. Loop over the results instead:

for res in extractIngredientInfo(ingredientLine, sectionTitle):
    yield res

Now you actually iterate over the nested generator function results, and pass them on to the caller (the consumer of the outer nested generator function).