Question

I recently saw the following like structure can contains info on a buffer. I know that the “buffer” field may point to the start address of the buffer, so we can use memcpy(pDst, pEntry->buffer, bufferLen) to copy the buffer to another place (pDst), but how can we assign Entry?

struct Entry
{
    // data fields.
    size_t bufferLen;
    unsigned char buffer[1];
};

Answer 1

This is an example of the old, pre-C99, ‘struct hack’ that was converted into a ‘flexible array member’ in C99. Suppose you want to store a string. You might dynamically allocate the space using:

struct Entry *make_entry(const char *str)
{
    size_t  len = strlen(str) + 1;
    struct Entry *e = malloc(sizeof(struct Entry) + len);
    if (e != 0)
    {
        e->bufferLen = len;
        strcpy(e->buffer, str);
    }
    return e;
}

In C99, you’d write the same code; the declaration of the structure would be different, though, as you do not specify the size of the array:

struct Entry
{
    size_t bufferLen;
    unsigned char buffer[];
};

This would use less space, especially on a 64-bit system where sizeof(size_t) == 8.

There are major limitations on such structures (both flexible array member style and struct hack style), one of which is that you can’t simply assign the structures. You have to allow for the extra space and copy it with memmove() or memcpy().

struct Entry *dup_entry(const struct Entry *e)
{
    struct Entry *r = malloc(sizeof(struct Entry) + e->bufferLen);
    if (r != 0)
        memmove(r, e, sizeof(struct Entry) + e->bufferLen);
    return r;
}

You also cannot create arrays of such structures (but you can create arrays of pointers to such structures).

---

I think in your make_entry() function, we can allocate one less memory as the Entry->buffer has already stored one char.

Interesting comment — and an understatement. At one time there was another answer (now deleted), and it collected some interesting and apposite comments, which I’m about to co-opt into this answer.

Michael Burr noted:

memcpy(pDst, pEntry, offsetof(struct Entry, buffer[pEntry->bufferLen])) would be a safer idiom — if there happens to be more than one entry before the array hack at the end of the struct, you don’t have to manually account for all of them, and it lets the compiler deal with any padding before the hack array automatically. Similarly, allocation of an instance can be: pEntry = malloc(offsetof(struct Entry, buffer[desiredVarArrayElements]))

This is an intriguing observation, and much in line with what you are suggesting. This use of offsetof() leads to results which are not compile-time constants; the result depends on the value used as the subscript at runtime. That’s unorthodox, but I can see nothing wrong with it. It would be crucial for the size used as a subscript to include the null byte in the count.

It also leads to the interesting observation that if this technique was used for short strings, you could end up allocating fewer bytes for the structure than sizeof(struct Entry) would. This is because the structure as a whole will be 4-byte aligned on most 32-bit machines and 8-byte aligned on most machines where sizeof(size_t) == 8 (generally 64-bit Unix systems, but not 64-bit Windows).

Thus, if the string to be allocated was only 3 bytes (2 characters and 1 null terminator byte), the size specified to malloc() would be 7 or 11 (on 32-bit or 64-bit machines respectively), compared with a structure size of 8 or 16.

So, the code I wrote for the ‘struct hack’ structure (as in the question) takes a (safe) shortcut and overallocates the total memory needed, probably by either 4 bytes or 8 bytes. It would be possible to reduce that overallocation. Note that the memory allocators themselves typically allocate a rounded up size — often (but not necessarily) a multiple of 8 bytes on 32-bit systems and 16 bytes on 64-bit systems. This means that an attempt to allocate fewer bytes might not have as much benefit as you’d expect (though it would have some benefit some of the time).

Note that the C99 flexible array member does not incur any wasted space at all; the size of the structure does not include the flexible array member at all. This makes it easier to use than the struct hack; you don’t have to worry about wasted space in the same way.

Steve Jessop noted:

If you’re making such structs copyable, then in any case you should probably have a capacity as well as a size field in the struct. Then write a function to copy them which checks the capacity, copies the size (but not the capacity), and copies the data. If you do those three things separately then you don’t have to worry about layout. And since there are are so many things to do, users of the struct shouldn’t try to copy it themselves, they should use the function.

This is, of course, exactly why I defined dup_entry(); it is indeed necessary to prevent people making a fool of themselves when copying. The make_entry() and dup_entry() functions must be implemented to agree with each other, but that’s not hard.