Question

I read buffer of data from somewhere to bytearray. Now, I want to work with this data using stream-like interface (i.e. read, seek etc.)

Can I just wrap my bytearray with io.BytesIO?

mybytearray = bytearray(...)
stream = io.BytesIO(mybytearray)

My fear here is BytesIO copies data of mybytearray, but I don’t want it – since buffer is very big. I don’t want copies, I want the stream to work on original data and can modify it too. What can be done?

Answer 1

BytesIO manages its own memory and will copy a buffer used to initialize it. You could encapsulate your bytearray in a file-like class. Or you can go the other way, letting the BytesIO object handle memory allocation. Then you can get a view of the buffer that can be modified by index and slice, but you can’t re-size the buffer while the view exists:

>>> f = io.BytesIO(b'abc')
>>> view = f.getbuffer()

>>> view[:] = b'def'
>>> f.getvalue()
b'def'

>>> view[3] = b'g'
IndexError: index out of bounds

>>> f.seek(0, 2)
>>> f.write(b'g')
BufferError: Existing exports of data: object cannot be re-sized

>>> del view
>>> f.write(b'g')
>>> f.getvalue()
b'defg'

---

Edit:

See issue 22003, BytesIO copy-on-write. The latest patch (cow6) supports copy-on-write for bytes only.