Question

Something crossed my mind recently in Python: x = y(z) is equivalent to x = y.__call__(z). However, a test appears to invalidate that assumption and also leads to Python’s interpreter to crash.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def ret(*args):
...     return args
...
>>> ret(1, 2, 3)
(1, 2, 3)
>>> for _ in range(1000000):
...     ret = ret.__call__
...
>>> ret(1, 2, 3)

Running the second ret(1, 2, 3) leads Python to crash and return to the command prompt (image).

1. What is happening in the background when the line ret = ret.__call__ executes?
2. Why does Python stop working on the last line, and should it be reported as a bug?

Useless Reference: Python functions and their __call__ attribute

Answer 1

You are creating a deeply nested structure of method wrappers. Each method wrapper still has a reference to self, where self is a reference to the parent method wrapper, all the way back to the original function:

>>> ret, ret.__call__.__self__
(<function ret at 0x10f17a050>, <function ret at 0x10f17a050>)
>>> ret.__call__, ret.__call__.__call__.__self__
(<method-wrapper '__call__' of function object at 0x10f17a050>, <method-wrapper '__call__' of function object at 0x10f17a050>)

Note how the memory address of the method wrapper __self__ attribute points to the parent object.

If you create enough of these wrappers, you could run out of memory.

Python creates such wrappers for all functions bound to an instance. It’s the same for a custom class with a method:

>>> class Foo: 
...     def bar(self): return
... 
>>> Foo().bar
<bound method Foo.bar of <__main__.Foo object at 0x10f1798d0>>
>>> Foo().bar, Foo().bar.__self__
(<bound method Foo.bar of <__main__.Foo object at 0x10f179710>>, <__main__.Foo object at 0x10f179850>)

Methods are created from functions as needed, when accessing the method via attribute access. Because they hold a reference to self, they are retained in memory as long as you hold a reference to them. Your chain of references thus holds 100000 memory wrappers alive.