Question

I’m a programming newbie and am having some trouble understanding an example from my python textbook (“Beginning Python” by Magnus Lie Hetland). The example is for a recursive generator designed to flatten the elements of nested lists (with arbitrary depth):

def flatten(nested):
    try:
        for sublist in nested:
            for element in flatten(sublist):
                yield element
    except TypeError:
        yield nested

You would then feed in a nested list as follows:

>>> list(flatten([[[1],2],3,4,[5,[6,7]],8]))
[1,2,3,4,5,6,7,8]

I understand how the recursion within flatten() helps to whittle down to the innermost element of this list, ‘1’, but what I don’t understand is what happens when ‘1’ is actually passed back into flatten() as ‘nested’. I thought that this would lead to a TypeError (can’t iterate over a number), and that the exception handling was what would actually do the heavy lifting for generating output… but testing with modified versions of flatten() has convinced me that this isn’t the case. Instead, it seems like the ‘yield element’ line is responsible.

That said, my question is this… how can ‘yield element’ ever actually be executed? It seems like ‘nested’ will either be a list – in which case another layer of recursion is added – or it’s a number and you get a TypeError.

Any help with this would be much appreciated… in particular, I’d love to be walked through the chain of events as flatten() handles a simple example like:

list(flatten([[1,2],3]))

Answer 1

I have added some instrumentation to the function:

def flatten(nested, depth=0):
    try:
        print("{}Iterate on {}".format('  '*depth, nested))
        for sublist in nested:
            for element in flatten(sublist, depth+1):
                print("{}got back {}".format('  '*depth, element))
                yield element
    except TypeError:
        print('{}not iterable - return {}'.format('  '*depth, nested))
        yield nested

Now calling

list(flatten([[1,2],3]))

displays

Iterate on [[1, 2], 3]
  Iterate on [1, 2]
    Iterate on 1
    not iterable - return 1
  got back 1
got back 1
    Iterate on 2
    not iterable - return 2
  got back 2
got back 2
  Iterate on 3
  not iterable - return 3
got back 3