Question

I am writing a script that will be polling a directory looking for new files.

In this scenario, is it necessary to do some sort of error checking to make sure the files are completely written prior to accessing them?

I don’t want to work with a file before it has been written completely to disk, but because the info I want from the file is near the beginning, it seems like it could be possible to pull the data I need without realizing the file isn’t done being written.

Is that something I should worry about, or will the file be locked because the OS is writing to the hard drive?

This is on a Linux system.

Answer 1

On Unix, unless the writing application goes out of its way, the file won’t be locked and you’ll be able to read from it.

The reader will, of course, have to be prepared to deal with an incomplete file (bearing in mind that there may be I/O buffering happening on the writer’s side).

If that’s a non-starter, you’ll have to think of some scheme to synchronize the writer and the reader, for example:

• explicitly lock the file;
• write the data to a temporary location and only move it into its final place when the file is complete (the move operation can be done atomically, provided both the source and the destination reside on the same file system).