Question

For a small to-be-embedded application, I wrote a few functions + struct that work as String Buffer (similar to std::stringstream in C++).

While the code as such works fine, There are a few not-so-minor problems:

• I never before wrote functions in C that manually allocate and use growing memory, thus I’m afraid there are still some quirks that yet need to be adressed
• It seems the code allocates far more memory than it actually needs, which is VERY BAD
• Due to warnings reported by valgrind I have switched from malloc to calloc in one place in the code, which sucessfully removed the warning, but I’m not entirely sure if i’m actually using it correctly

Example of what I mean that it allocates more than it really needs (using a 56k file):

==23668== HEAP SUMMARY:
==23668==     in use at exit: 0 bytes in 0 blocks
==23668==   total heap usage: 49,998 allocs, 49,998 frees, 1,249,875,362 bytes allocated

… It just doesn’t look right …

The code in question is here (too large to copy it in a <code> field on SO): http://codepad.org/LQzphUzd

Help is needed, and I’m grateful for any advice!

Answer 1

The way you are growing your buffer is rather inefficient. For each little piece of string, you realloc() memory, which can mean new memory is allocated and the contents of the “old” memory are copied over. That is slow and fragments your heap.

Better is to grow in fixed amounts, or in fixed percentages, i.e. make the new size 1.5 or 2 times the size of the old size. That also wastes some memory, but will keep the heap more usable and not so many copies are made.

This means you’ll have to keep track of two values: capacity (number of bytes allocated) and length (actual length of the string). But that should not be too hard.

I would introduce a function “FstrBuf_Grow” which takes care of all of that. You just call it with the amount of memory you want to add, and FstrBuf_Grow will take care that the capacity matches the requirements by reallocing when necessary and at least as much as necessary.

...

void FstrBuf_Grow(FstringBuf *buf, size_t more)
{
    while (buf->length + more) > buf->capacity
        buf->capacity = 3 * buf->capacity / 2;
    buf->data = realloc(buf->data, buf->capacity + 1);
}            

That multiplies capacity by 1.5 until data is large enough. You can choose different strategies, depending on your needs.