Question

I have an iterator that returns context managers.

I want a pythonic with statement, that emulates the behaviour of several nested with statements, one for each context manager returned by the iterator.

One could say, I want a generalisation of the (deprecated) contextlib.nested function.

Answer 1

contextlib.nested has two major problems that caused it to be deprecated.

1. first problem is that inner context managers might raise Exceptions during __init__ or __new__, and these exceptions would cause the whole with statement to abort without calling __exit__ of the outer manager.
2. The second problem is more complicated. If one of the inner managers raises an exception and one of the outer managers catches it by returning True in __exit__, the block should still be executed. But in the implementation of nested, it just raises a RuntimeError without executing the block. This problem probably requires a total rewrite of nested.

But it is possible to solve the first problem by just removing one * in the definition of nested!
This changes the behaviour such that nested doesn’t accept argument lists anymore (which isn’t useful anyway because with can handle that already) but only an iterator. I therefore call the new version “iter_nested“.
The user can then define an iterator that instantiates the context managers during iteration.

An example with a generator:

def contexts():
    yield MyContext1()
    yield MyContext2()

with iter_nested(contexts()) as contexts:
    do_stuff(contexts[0])
    do_other_stuff(contexts[1])

The difference between the codes of the original and my changed version of nested is here:

from contextlib import contextmanager

@contextmanager
--- def nested(*managers):
+++ def iter_nested(mgr_iterator):
    --- #comments & deprecation warning
    exits = []
    vars = []
    --- exc = (None, None, None)
    +++ exc = None # Python 3
    try:
        --- for mgr in managers:
        +++ for mgr in mgr_iterator:
            exit = mgr.__exit__
            enter = mgr.__enter__
            vars.append(enter())
            exits.append(exit)
        yield vars
# All of the following is new and fit for Python 3
except Exception as exception:
    exc = exception
    exc_tuple = (type(exc), exc, exc.__traceback__)
else:
    exc_tuple = (None, None, None)
finally:
    while exits:
        exit = exits.pop()
        try:
            if exit(*exc_tuple):
                exc = None
                exc_tuple = (None, None, None)
        except Exception as exception:
            exception.__context__ = exc
            exc = exception
            exc_tuple = (type(exc), exc, exc.__traceback__)
    if exc:
        raise exc