Question

Note : if you know of any (non-elaborate) library code that does what I want please enlighten a C/C++ programmer, I’ll accept that as answer.

I have a global variable set to an instance of the following class. It’s purpose is to allow me to set some manual interruption points to place some quick and dirty printf style debug points in a scrapy spider (I specifically need to break when certain criteria are met to tune a parser, there are some extremely rare input data anomalies) — Adapted from this.

Os is OS X 10.8.

import termios, fcntl, sys, os

class DebugWaitKeypress(object):
    def __init__(self):
        self.fd = sys.stdin.fileno()
        self.oldterm = termios.tcgetattr(self.fd)
        self.newattr = termios.tcgetattr(self.fd)
        self.newattr[3] = self.newattr[3] & ~termios.ICANON & ~termios.ECHO
        termios.tcsetattr(self.fd, termios.TCSANOW, self.newattr)

        self.oldflags = fcntl.fcntl(self.fd, fcntl.F_GETFL)
        fcntl.fcntl(self.fd, fcntl.F_SETFL, self.oldflags | os.O_NONBLOCK)

    def wait(self):
        sys.stdin.read(1)

    def __del__(self):
        print "called del"
        termios.tcsetattr(self.fd, termios.TCSAFLUSH, self.oldterm)
        fcntl.fcntl(self.fd, fcntl.F_SETFL, self.oldflags)

When I press Ctrl-C and the process is unwinding I get the following exception :

Exception AttributeError: "'NoneType' object has no attribute 'tcsetattr'" in <bound method DebugWaitKeypress.__del__ of <hon.spiders.custom_debug.DebugWaitKeypress object at 0x108985e50>> ignored

I’m missing something about the mechanics of object lifetimes I guess ? How do remedy the situation. AFAIK any class instances should be destroyed before the imported code does, no ? in reverse order of declaration/definition.

I would just ignore this if the terminal wasn’t screwed up after the process exits 😀

edit:

Delian’s comment on seth’s answer led me to understand that I need to use a C main() like function, or any other function/generator which dominates as a root function and initialise the context there. This way When the process is going down the __exit__ method of the context manager will get called. And I won’t have to reprogram the terminal stream on each wait() call.

Although the cost of the reprogramming is potentially immaterial, it is good to know how one would these essential C/C++ semantics in python.

edit 2:

Twisted (which scrapy uses) goes apeshit when messing with stdin. So I had to solve the problem with file IO.

Answer 1

Long story short: __del__ is useless for this purpose (and pretty much any other purpose; you should probably forget it exists). If you want deterministic cleanup, use a context manager.

AFAIK any class instances should be destroyed before the imported code does, no ? in reverse order of declaration/definition.

That’s C++. Forget it. Python does not care about this, in fact it does not even care about most things which are requirements for doing it. There is no such thing as a declaration in the entire Python language, and module-level variables are stored in what is essentially an unordered associative array. Variables do not store objects, they store references (which are not C++ references, they’re basically pointers without pointer arithmetic) — objects are on the heap and don’t know a thing about variables, bindings, statements, or order of statements.

Moreover, when an objects is garbage collected, and whether it is gc’d at all, is undefined. You get a mostly deterministic picture in CPython due to reference counting, but even there it falls down the second you have cycles. The consequence is that __del__ may be called at any point in time (including when half of the module is already torn down) or not at all. Multiple objects defining __del__ referencing each other are also trouble, although some GCs try hard to do the right thing.

Bottom line is, you can assume very little at the time __del__ runs, so you can’t do very much. You get a last shot at disposing resources that should have been cleaned up via another method but weren’t, and that’s pretty much it. Rule of thumb: Never rely on it for anything mandatory.

Instead, create a context manager and use it via with. You get deterministic cleanup, without worrying about object lifetime. Because, truth to be told, object lifetime and resource lifetime are two entirely different things, and only entangled in C++ because it’s the best way to do resource management in that environment. In Python, RAII does not apply, instead we have this:

with <context manager> as var:
    # do something
# "context closed", whatever that means - for resources, usually cleanup

By the way, you can define it far more conveniently via contextlib (quickly transliterated from your version, may contain errors or ugliness):

from contextlib import contextmanager


@contextmanager
def debug_wait_keypress():
    fd = sys.stdin.fileno()
    oldterm = termios.tcgetattr(fd)
    newattr = termios.tcgetattr(fd)
    newattr[3] = newattr[3] & ~termios.ICANON & ~termios.ECHO
    termios.tcsetattr(fd, termios.TCSANOW, newattr)
    oldflags = fcntl.fcntl(fd, fcntl.F_GETFL)
    fcntl.fcntl(fd, fcntl.F_SETFL, oldflags | os.O_NONBLOCK)
    try:
        yield
    finally:
        termios.tcsetattr(fd, termios.TCSAFLUSH, oldterm)
        fcntl.fcntl(fd, fcntl.F_SETFL, oldflags)

Your wait method becomes a free function.