Question

I am running a django app that includes matplotlib and allows the user to specify the axes of the graph. This can result in ‘Overflow Error: Agg complexity exceeded’

When that happens up to 100MB of RAM get tied up. Normally I free that memory up using fig.gcf(), plot.close(), and gc.collect(), but the memory associated with the error does not seem to be associated with the plot object.

Does anyone know how I can release that memory?

Thanks.

Here is some code that gives me the Agg Complexity Error.

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np      
import gc

a = np.arange(1000000)
b = np.random.randn(1000000)

fig = plt.figure(num=1, dpi=100, facecolor='w', edgecolor='w')
fig.set_size_inches(10,7)
ax = fig.add_subplot(111)
ax.plot(a, b)

fig.savefig('yourdesktop/random.png')   # code gives me an error here

fig.clf()    # normally I use these lines to release the memory
plt.close()
del a, b
gc.collect()

Answer 1

I assume you can run the code you posted at least once. The problem only manifests itself after running the posted code many times. Correct?

If so, the following avoids the problem without really identifying the source of the problem.
Maybe that is a bad thing, but this works in a pinch: Simply use multiprocessing to run the memory-intensive code in a separate process. You don’t have to worry about fig.clf() or plt.close() or del a,b or gc.collect(). All memory is freed when the process ends.

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np      

import multiprocessing as mp

def worker():
    N=1000000
    a = np.arange(N)
    b = np.random.randn(N)

    fig = plt.figure(num=1, dpi=100, facecolor='w', edgecolor='w')
    fig.set_size_inches(10,7)
    ax = fig.add_subplot(111)
    ax.plot(a, b)

    fig.savefig('/tmp/random.png')   # code gives me an error here

if __name__=='__main__':
    proc=mp.Process(target=worker)
    proc.daemon=True
    proc.start()
    proc.join()

You don’t have to proc.join() either. The join will block the main process until the worker completes. If you omit the join, then the main process simply continues with the worker process working in the background.