Question

I have two binary input files, firstfile and secondfile. secondfile is firstfile + additional material. I want to isolate this additional material in a separate file, newfile. This is what I have so far:

import os
import struct

origbytes = os.path.getsize(firstfile)
fullbytes = os.path.getsize(secondfile)
numbytes = fullbytes-origbytes

with open(secondfile,'rb') as f:
    first = f.read(origbytes)
    rest = f.read()

Naturally, my inclination is to do (which seems to work):

with open(newfile,'wb') as f:
    f.write(rest)

I can’t find it but thought I read on SO that I should pack this first using struct.pack before writing to file. The following gives me an error:

with open(newfile,'wb') as f:
    f.write(struct.pack('%%%ds' % numbytes,rest))

-----> error: bad char in struct format

This works however:

with open(newfile,'wb') as f:
    f.write(struct.pack('c'*numbytes,*rest))

And for the ones that work, this gives me the right answer

with open(newfile,'rb') as f:
    test = f.read()

len(test)==numbytes

-----> True

Is this the correct way to write a binary file? I just want to make sure I’m doing this part correctly to diagnose if the second part of the file is corrupted as another reader program I am feeding newfile to is telling me, or I am doing this wrong. Thank you.

Answer 1

There is no reason to use the struct module, which is for converting between binary formats and Python objects. There’s no conversion needed here.

Strings in Python 2.x are just an array of bytes and can be read and written to and from files. (In Python 3.x, the read function returns a bytes object, which is the same thing, if you open the file with open(filename, 'rb').)

So you can just read the file into a string, then write it again:

import os

origbytes = os.path.getsize(firstfile)
fullbytes = os.path.getsize(secondfile)
numbytes = fullbytes-origbytes

with open(secondfile,'rb') as f:
    first = f.seek(origbytes)
    rest = f.read()

with open(newfile,'wb') as f:
    f.write(rest)