Question

I’ve been running Python scripts that make several calls to some functions, say F1(x) and F2(x), that look a bit like this:

x = LoadData()

for j in range(N):
    y = F1(x[j])
    z[j] = F2(y)

    del y

SaveData(z)

Performance is a lot faster if I keep the “del y” line. But I don’t understand why this is true. If I don’t use “del y”, then I quickly run out of RAM and have to resort to virtual memory, and everything slows to a crawl. Buy if I use “del y”, then I am repeatedly flushing and re-allocating the memory for y. What I would like to do is have y sit as static memory, and reuse the memory on every F1(x) call. But from what I can tell, that isn’t what’s happening.

Also, not sure if it’s relevant, but my data consists of numpy arrays.

Answer 1

Without the del y you might need twice as much memory. This is because for each pass through the loop, y is bound to the previous value of F1 while the next one is calculated.

once F1 returns y is rebound to that new value and the old F1 result can be released.

This would mean that the object returned by F1 occupies quite a lot of memory

Unrolling the loop for the first couple of iterations would look like this

y = F1(x[0])   # F1(x[0]) is calculated, then y is bound to it
z[j] = F2(y)
y = F1(x[1])   # y is still bound to F1(x[0]) while F1(x[1]) is computed
               # The memory for F1(X[0]) is finally freed when y is rebound
z[j] = F2(y)

using del y is a good solution if this is what is happening in your case.