Question

Having been told by most of the savvy students around campus here, binary file i/o is always faster than formatted i/o. After a bit of googling, i determined that the most efficient way of accomplishing binary i/o was not by streaming directly from the file, each value at a time, but by reading in a large chunk of data, like 4k or 8k chunks, and then parse the data out of that buffer. my question is, say that i used fstream to grab a chunk of data out of a file and stored it into a char array of 8192 bytes. now let’s say that it had a header of about 30 bytes, and then all the data after that were like 72 byte stuctures apiece. This would leave 56 bytes of incomplete data at the end which i couldn’t read into a structure. How do i handle this to maximize efficiency (yes, using c++ streams, please no whining about c++ streams here, i’ve heard dozens of arguments on it. This is wholly a question on binary i/o and NOT the efficacy of specific mediums of i/o please). Do i just leave that 56 bytes at the end unused, or do i try to frankenstein together the tail-end and first 16 bytes of the next buffer, or is there still another way to approach this? My question is also, WHY is reading chunks into a buffer like this before parsing, faster? Conversely, if you do not believe this is true, then please explain why if you can. Thanks for your time.

Answer 1

Use the standard iostream library, it performs buffering automatically.