I have a data source I’m working with in Python. I’d like to save that data to a files such that once a threshold is hit (ie: 1K, 1M) the file is closed and a new file is automatically opened to save the data.
ie:
<file handler with buffer 200>
file.write('a'*1000)
The line above would generate 5 files based on the data. Is there a pre-built python library that will handle this, or do I need to write one myself?
If a logger framework is too much, you can do it yourself — shouldn’t need more than a dozen lines of code or so. The easiest way to get the size of your file is by calling the
tell()method of your open file descriptor.You could also keep track of the bytes being output, but this requires additional logic if your program sometimes appends to a pre-existing file.