Question

I am polling a directory constantly for files and every time I see a file that meets some certain criteria for reading, I open the file and parse it.

string newLine;

ifstream fileReader;

fileReader.open(filename.c_str());

while(!fileReader.eof())
{
getline(fileReader, newLine);
    // do some stuff with the line...
}
filereader.close();

The above code is in a loop that runs every 1 second checking a directory for new files. My issue is that as I am transferring files into the folder for processing, my loop finds the file and passes the name of the file to ifstream who then opens it and tries to parse an incomplete file. How do I make ifstream wait until the file is done being written before it tries to parse the file?

EDIT:
Wanted to better word the issue here since a replier seems to have misunderstood my issue. I have 2 directories

mydirectory/
mydirectoryParsed/

THe way my code works is that my program checks for files in mydirectory/ and when it finds them, parses them and uses the information in the files. No writing to the files are done. Once I am done parsing the file, the file is moved to mydirectoryparsed/

The issue is that when I transfer files over the network into mydirectory/ the ifstream sees these files midtransfer and starts reading them before they finish writing to the directory. How do I make ifstream wait until the file is completely written before parsing it?

Answer 1

Don’t transfer the files directly into the directory that your program is watching; instead, transfer them into a different directory on the same drive, and then when the transfer is done, move them into the watched directory. That way, the complete file appears in the watched directory in a single atomic operation.

Alternatively, you could use a naming convention in the watched directory — append a suffix like “.partial” to files that are being transferred, and then rename the file to remove the suffix when the transfer is done. Have your program ignore files whose names end with the suffix.