Question

In a Python program I’m writing, I’ve built up a linked list using a dictionary which maps each node to its successor (with the last node mapped to None).

(Actually, the dictionary holds what Wikipedia tells me is called a spaghetti stack, which is a tree where each node is linked to its parent but not its children. This means there are many partially-overlapping paths from the leaf nodes to the root node. I only care about one of those paths, starting from a specific leaf node. This doesn’t really matter for the question, other than to rule out any solution that involves iterating over all the elements in the dictionary.)

I need to pass this list to another function as an iterable. I know I can do this using a generator function (see code below), but it seems like there ought to be a built-in function to make the iterator I need in one line (or a perhaps a generator expression). I’ve done a bit of searching through the documentation, but there’s nothing in the itertools or functools modules seems applicable, and I’m not sure where else to look.

Here’s the generator function I have now. The outer function can be eliminated (inlined) but the inner generator seems to be the only simple way to make an iterable for the data:

def makeListGenerator(nextDict, start):
    def gen(node):
        while node:
            yield node
            node = nextDict[node]
    return gen(start)

It seems like there should be a pattern for this sort of generator, but I’m not sure what it would be called. Here’s a generic version:

def makeGenericGenerator(nextFunc, continueFunc, start):
    def gen(value):
         while continueFunc(value):
             yield value
             value = nextFunc(value)
    return gen(start)

I could implement the specific version using this one using the call:

makeGenericGenerator(lambda v: nextDict[v], bool, start)

Does something like that already exist in the Python standard library?

Answer 1

The essential problem you face is that every time another value is taken from your iterable, your iterable has to remember that value, so that it knows how to generate the next value. In other words, your iterable needs to maintain its own state.

That means that, whether or not there’s a good answer to your question, using a generator is probably the right solution — because that’s exactly what generators were made for! The whole point of generators is that they save their state between calls to their next method, and that’s exactly what you need.

Generator expressions, on the other hand, are better for stateless transformations. A lot of people try to shoehorn state into them, and it’s generally kind of ugly. I actually spent a little while trying to do it for your case, and couldn’t get a generator expression to work. I finally found something that sort of works. This uses the little-known callable_iterator version of iter:

>>> d = {1:2, 2:3, 3:4, 4:5, 5:None}
>>> list(iter(lambda st=[1]: st.__setitem__(0, d[st[0]]) or st[0], None))
[2, 3, 4, 5]

I present this not as an answer, but as a demonstration of why this doesn’t work very well.