Question

I am working on a project which basically involves parsing and storing information about a large set of directories and files and gives me a summary of the number of files that were newly added (More like a ‘File Sniffer’).

The result of my program is that when I run it on say some 30 GB of data, I see the memory (working set) being allocated grow up to 700,000 K (memory leak).

In, my code, I am using quite a lot of object creation and destruction. The most used object is quite heavy.

Any suggestions as what could be done to reduce the memory usage. Please let me know if anything is unclear.

The main part of the code is something like this:

void analyzeHelper(std::string path){
    directory *dir=new directory();
    if(!dir->DirectoryExists(path))
        return;
    dir->SetCurrentDirectory(path);
    summary=new Summary();                        // Heavy class
    for(size_t i=0;i<dir->GetDirectories("*.*").size();i++)
        summary->addChild(dir->GetDirectories("*.*")[i]);
    for(size_t i=0;i<dir->GetFiles().size();i++)
        summary->addFile(Path<std::string>::toLower(dir->GetFiles()[i]));
    summary->setLocation(path);
    delete dir;
    compare(); //  here I run a comparision of the same directory with its previous version
    createMetadata(); // create an xml file to store the current version
    int size=summary->childSize();
    std::vector<std::string> _children=summary->getChildren();
    delete summary;
    for(int i=0;i<size;i++)
    {
        FileSniffer *f=new FileSniffer(xmlroot);
        f->setRoot(dirRoot,path,remroot);
        f->analyzeHelper(_children[i]);
        delete f;
    }

}

A related question: How bad is it to keep copying vectors? (As I am doing with ‘_children’ in the code)

Answer 1

There is a memory leak if a directory does not exist:

directory *dir=new directory();
if(!dir->DirectoryExists(path))
    return;

dir must be deleted before returning (a smart pointer would simplify the management of dir) or just allocate dir on the stack:

directory dir;
if(!dir.DirectoryExists(path))
    return;