Question

I’ve written a simple library file with a function for reading lines from a file of any size. The function is called by passing in a stack-allocated buffer and size, but if the line is too big, a special heap-allocated buffer is initialized and used to pass back a larger line.

This heap-allocated buffer is function-scoped and declared static, initialized to NULL at the beginning of course. I’ve written in some checks at the beginning of the function, to check if the heap buffer is non-null; if this is the case, then the previous line read was too long. Naturally, I free the heap buffer and set it back to NULL, thinking that the next read will likely only need to fill the stack-allocated buffer (it should be very rare to see lines over 1MB long, even in our application!).

I’ve gone over the code and tested it fairly thoroughly, both by reading it carefully and by running a few tests. I am reasonably confident that the following invariant is maintained:

• The heap buffer will be null (and will not leak any memory) on function return if the stack buffer is all that is needed.
• If the heap buffer is not null, because it was needed, it will be freed on the next function call (and possibly reused if needed on that next line).

But I’ve thought of a potential problem: If the last line in a file is too long, then since the function is presumably not called again, I’m not sure I have any way to free the heap buffer– it is function-scoped, after all.

So my question is, how do I go about freeing dynamically allocated memory in a function-scoped static pointer, ideally without calling the function again? (And ideally without making it a global variable, either!)

Code available on request. (I just haven’t got access now, sorry. And I’m hoping the question is sufficiently general and well-explained for it not to be needed, but by all means feel free to disabuse me of that notion!)

---

EDIT: I feel I should add a couple of notes about the usage of the function.

This particular function is used in the form of lines being read serially from a file, and then immediately copied into POD structs, one line per struct. Those are created on the heap as the file is read, and each one of those structs has a char pointer containing (a cleaned up version of) a line from the file. In order for these to persist, a copy already has to occur. (That was one of the big counterarguments brought up in many of the answers– oh no, the line needs to be COPIED, oh dearie me).

As for multithreading, as I said this is designed to be used serially. No, it isn’t thread safe, but I don’t care.

Thanks for the multitude of responses, though! I’ll read them more thoroughly when I get time. Currently, I’m leaning towards either passing an extra pointer around or redesigning the function so that when fgets shows EOF, then I might just build the freeing logic there instead and the user hopefully won’t need to worry about it.

Answer 1

If you can change the function, I would recommend changing the function interface itself. I know you have spent a lot of time debugging and testing it, but there are a few problems with your current implementation:

• it is not thread-safe,
• the user has no control over the data, so he must copy it if he needs it later, most likely in a buffer that’s going to be malloc()ed, thus nullifying any advantage you got by the selective use of malloc() in your function,
• most importantly, as you have discovered, a special action has to be taken by the user for a long last line.

Your users should not be worried by the implementation oddity of your function, they should be able to “just use it”.

Unless you are doing it for educational purposes, I would recommend looking at this page, which has one implementation of “reading an arbitrarily long line from a stream”, and links to other such implementations (each implementation is slightly different from the others, so you should be able to find one that you like).

Based upon your edit, MT-safe is not a requirement, and a copy is always going to happen. So, the most obvious design is one of the two:

• Let the user supply a char **, which points to a buffer that your function will allocate, using a combination of malloc() and realloc() (if needed). It is the user’s responsibility to free() it when done. That way, the user doesn’t have to copy the data again, since he can pass a pointer to wherever the final destination of the data is.
• return a char * that is allocated by your function. Again, it’s the user’s responsibility to free() it.

Both are pretty much equivalent.

For your current implementation, you can always return “not end of file” if the last line is very long, and doesn’t end in a newline. Then, the user is going to call your function again, and then you can free your buffer. Personally, I would be happier with a function that allows me to read as many lines as I want, and not force me to go to the end of file.