Question

I have a program that generates files containing random distributions of the character A – Z. I have written a method that reads these files (and counts each character) using fread with different buffer sizes in an attempt to determine the optimal block size for reads. Here is the method:

int get_histogram(FILE * fp, long *hist, int block_size, long *milliseconds, long *filelen)
{
    char *buffer = new char[block_size];
    bzero(buffer, block_size);

    struct timeb t;
    ftime(&t);
    long start_in_ms = t.time * 1000 + t.millitm;

    size_t bytes_read = 0;
    while (!feof(fp))
    {
        bytes_read += fread(buffer, 1, block_size, fp);
        if (ferror (fp))
        {
            return -1;
        }
        int i;
        for (i = 0; i < block_size; i++)
        {
            int j;
            for (j = 0; j < 26; j++)
            {
                if (buffer[i] == 'A' + j)
                {
                    hist[j]++;
                }
            }
        }
    }

    ftime(&t);
    long end_in_ms = t.time * 1000 + t.millitm;
    *milliseconds = end_in_ms - start_in_ms;
    *filelen = bytes_read;

    return 0;
}

However, when I plot bytes/second vs. block size (buffer size) using block sizes of 2 – 2^20, I get an optimal block size of 4 bytes — which just can’t be correct. Something must be wrong with my code but I can’t find it.

Any advice is appreciated.

Regards.

EDIT:

The point of this exercise is to demonstrate the optimal buffer size by recording the read times (plus computation time) for different buffer sizes. The file pointer is opened and closed by the calling code.

Answer 1

There are many bugs in this code:

• It uses new[], which is C++.
• It doesn’t free the allocated memory.
• It always loops over block_size bytes of input, not bytes_read as returned by fread().

Also, the actual histogram code is rather inefficient, since it seems to loop over each character to determine which character it is.

UPDATE: Removed claim that using feof() before I/O is wrong, since that wasn’t true. Thanks to Eric for pointing this out in a comment.