Question

I could use a set of eyes (or more) on this code. I’m trying to read in a set amount of bytes from a filestream (f1) to an array/buffer (file is a text file, array is of char type). If I read in size “buffer – 1” I want to “realloc” the array and the continue to read, starting at where I left off. Basically I’m trying to dynamically expand the buffer for the file of unknown size. What I’m wondering:

1. Am I implementing this wrong?
2. How would I check failure conditions on something like “realloc”
   with the code the way it is?
3. I’m getting a lot of warnings when I compile about “implicit declaration of built-in function realloc…” (I’m seeing that warning for my use of read, malloc, strlen, etc. as well.
4. When “read()” get’s called a second time (and third, fourth, etc.) does it read from the beginning of the stream each time? That could be my issue is I only seem to return the first “buff_size” char’s.

Here’s the snippet:

//read_buffer is of size buff_size
n_read = read(f1, read_buffer, buff_size - 1);
read_count = n_read;
int new_size = buff_size;
while (read_count == (buff_size - 1))
{

        new_size *= 2;
        read_buffer = realloc(read_buffer, new_size);
        n_read = read(f1, read_buffer[read_count], buff_size - 1);
        read_count += n_read;
}

As I am learning how to do this type of dynamic read, I’m wondering if someone could state a few brief facts about best practices with this sort of thing. I’m assuming this comes up a TON in the professional world (reading files of unknown size)? Thanks for your time. ALSO: As you guys find good ways of doing things (ie a technique for this type of problem), do you find yourselves memorizing how you did it, or maybe saving it to reference in the future (ie is a solution fairly static)?

Answer 1

If you’re going to expand the buffer for the entire file anyway, it’s probably easiest to seek to the end, get the current offset, then seek back to the beginning and read in swoop:

size = lseek(f1, 0, SEEK_END); // get offset at end of file

lseek(f1, 0, SEEK_SET); // seek back to beginning

buffer = malloc(size+1); // allocate enough memory.

read(f1, buffer, size);  // read in the file

Alternatively, on any reasonably modern POSIX-like system, consider using mmap.