Question

The code needs to find and process files that have not yet been processed on embedded-linux.
getDir is used to return the sorted contents of a directory.
The code below works perfectly after processing several few dozen or maybe more than 100 files but then dies with oom-killer.
Is this a bad way to use c++ vectors (loop vector inside loop vector)?
Might this method be causing the oom-killer?
Is there another approach that might work without blowing up?
Shouldn’t each vector be destroyed as it goes out of scope?
Does new/delete need to be used instead?
Also: valgrind for finding memory leaks is not integrated in the sdk for this processor (TI DM368) but the code is very short and there are no new statements. Note: the actual code checks an sql database for files that have already been processed but this code still caused oom-killer with the sql code commented out so it has been left out for simplicity. The file path format is /YYYYmmdd/HH/MMSS.SS.ext.

void getDir (string dir, vector<string> &files) {
    ...
    while ((dirp = readdir(dp)) != NULL) {
        files.push_back(string(dirp->d_name));
    closedir(dp);
    sort(files.begin(), files.end());

while (true) {
    vector<string> days;
    getDir(database_location, days);
    for (uint d=0; d<days.size(); d++) {
        vector<string> hours;
        getDir(database_location+days[d], hours);
        for (uint h=0; h<hours.size(); h++) {
            vector<string> files;
            string dir = database_location+days[d]+"/"+hours[h];
            getDir(dir, files);
            for (uint f=0; f<files.size(); f++) process(dir, files[f]);

Answer 1

1. Linux OOM killer is notoriously dumb. By default, don’t start blaming your own code.
2. You only have three vectors, of reasonable sizes. (If the vector size is unreasonable, it would be because the number of files in a single dir is also unreasonable).
3. You don’t need new/delete.

Since you’re not going to be running multi-threaded, consider making the three vectors static. Hack, but might just work.