Question

I am using Python to write chunks of text to files in a single operation:

open(file, 'w').write(text)

If the script is interrupted so a file write does not complete I want to have no file rather than a partially complete file. Can this be done?

Answer 1

Write data to a temporary file and when data has been successfully written, rename the file to the correct destination file e.g

with open(tmpFile, 'w') as f:
    f.write(text)
    # make sure that all data is on disk
    # see http://stackoverflow.com/questions/7433057/is-rename-without-fsync-safe
    f.flush()
    os.fsync(f.fileno())    
os.replace(tmpFile, myFile)  # os.rename pre-3.3, but os.rename won't work on Windows

According to doc http://docs.python.org/library/os.html#os.replace

Rename the file or directory src to dst. If dst is a non-empty directory, OSError will be raised. If dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail if src and dst are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement).

Note:

• It may not be atomic operation if src and dest locations are not on same filesystem

• os.fsync step may be skipped if performance/responsiveness is more important than the data integrity in cases like power failure, system crash etc