Question

I realized there is a memory leak in one python script. Which occupied around 25MB first, and after 15 days it is more than 500 MB.

I followed many different ways, and not able to get into the root of the problem as am a python newbie…

Finally, I got this following

objgraph.show_most_common_types(limit=20)
tuple                      37674
function                   9156
dict                       3935
list                       1646
wrapper_descriptor         1468
weakref                    888
builtin_function_or_method 874
classobj                   684
method_descriptor          551
type                       533
instance                   483
Kind                       470
getset_descriptor          404
ImmNodeSet                 362
module                     342
IdentitySetMulti           333
PartRow                    331
member_descriptor          264
cell                       185
FontEntry                  170

I set a break point, and after every iteration this is what is happening…

objgraph.show_growth()
tuple    37674       +10

What is the best way to proceed ?

(Pdb) c
(Pdb) objgraph.show_growth()
tuple    37684       +10

I guess printing out all the tuples, and cross checking — what this 10 tuple gets added everytime will give me some clue ? Kindly let me know how to do that..

Or is there any other way to find out this memory leak. Am using python 2.4.3, and because of many other product dependencies – Unfortunately i cannot / should not upgrade.

Answer 1

Am I reading correctly that the same script is running for 15 days non-stop?

For such long-running processes periodic restart is a good practice and it’s much easier to do than eliminating all memory leaks.

Update: Look at this answer, it seems to do exactly what you need — print all newly added objects that were not garbage collected.