Question

Below are two programs that write 50,000,000 bytes to a file.

The first program, written in C, utilizes a buffer, that once filled to a arbitrary value, writes to disk, and then repeats that process until all 50,000,000 bytes are written. I noticed that as I increased the size of the buffer, the program took less time to run. For instance, at BUFFER_SIZE = 1, the program took ~88.0463 seconds, whereas at BUFFER_SIZE = 1024, the program only took ~1.7773 seconds. The best time I recorded was when BUFFER_SIZE = 131072. As the BUFFER_SIZE increased higher than that, I noticed that it began to actually take a little longer.

The second program, written in C++, utilizes ofstream to write one byte at a time. To my surprise, the program only took ~1.87 seconds to run. I expected it to take a minute or so, like the C program with BUFFER_SIZE = 1. Obviously, the C++ ofstream handles file writing differently than I thought. According to my data, it is performing pretty similarly to the C file with BUFFER_SIZE = 512. Does it use some sort of behind-the-scenes buffer?

Here is the C program:

const int NVALUES = 50000000; //#values written to the file
const char FILENAME[] = "/tmp/myfile";
const int BUFFER_SIZE = 8192; //# bytes to fill in buffer before writing

main()
{
    int fd;  //File descriptor associated with output file
    int i;
    char writeval = '\0';
    char buffer[BUFFER_SIZE];

    //Open file for writing and associate it with the file descriptor
    //Create file if it does not exist; if it does exist truncate its size to 0
    fd = open(FILENAME, O_WRONLY|O_CREAT|O_TRUNC, S_IRUSR|S_IWUSR);

    for(i=0;i<NVALUES;i++)
    {
        //Package bytes into BUFFER_SIZE chunks 
                //and write a chunk once it is filled
        buffer[i%BUFFER_SIZE] = writeval;
        if((i%BUFFER_SIZE == BUFFER_SIZE-1 || i == NVALUES-1))
            write(fd, buffer, i%BUFFER_SIZE+1);

    }

    fsync(fd);

    close(fd);
}

Here is the C++ program:

int main()
{
    ofstream ofs("/tmp/iofile2");
    int i;

    for(i=0; i<50000000; i++)
        ofs << '\0';

    ofs.flush();
    ofs.close();

    return 0;
}

Thank you for your time.

Answer 1

Yes, ostreams use a stream buffer, some subclass of an instantiation of the template basic_streambuf. The interface of basic_streambuf is designed so that an implementation can do buffering if there’s an advantage in that.

However this is a quality of implementation issue. Implementations are not required to do this but any competent implementation will.

You can read all about it in chapter 27 of the ISO standard, though maybe a more readable source is The C++ Standard Library: A Tutorial and Reference (google search).