Question

I have a program which reads all the file system file/dir names, size etc. and populate in a tree data structure. Once this is done, it will generate a report.

I want write my program to collect and then report this data using memory in the most efficient way and without exceeding my heap space.

I worry that if the file system has a lot of files and dirs., it will consume a lot of memory and might eventually run out (malloc() will start to fail).

Eventually this is genuine memory consumption, Is there any methods/techniques to overcome this?

Answer 1

You could employ the Flyweight Design Pattern for each folder node.

http://en.wikipedia.org/wiki/Flyweight_pattern

Instead of storing the full path for each item, you could have a bit array of pointers to partial paths (folder names). These could then be easily reconstructed if needed.

It also depends on what you need for your report. Do you need to hold all the information in memory during construction, or could you just accumulate some of the space count variables as you traverse the tree?