Question

This is a theoretical question as I don’t have an actual problem, but I got to wondering …

If I had a huge file, say many gigs long and I wanted to change a single byte and I knew the offset of that byte, how could I do this efficiently? Is there a way to do this without rewriting the entire file and only writing the single byte?

I’m not seeing anything in the Python file api that would let me write to a particular offset in a file.

Answer 1

As long as you don’t need to insert or delete bytes, you can open the file in "r+" mode, use the seek method to position the file object at the byte to change, and write out one byte.

It may be more efficient to use the lower-level os.open, os.lseek, os.read, and os.write operations, which do not do any application-level buffering.

If you do need to insert or delete bytes, sorry, you’re out of luck: there is no way to do that without rewriting the entire file (from the point of the first insertion or deletion). This is a limitation of the POSIX (and AFAIK also Windows) low-level file APIs, not of Python specifically.