Question

Does anybody know how much overhead per single item is required?
When I insert 1 byte key the ‘stats’ command shows 65 bytes being consumed per item.

Is this expected and fixed?

Thanks,
Piotr

Answer 1

Yes, Memcache’s own data structures eat up more than 50 byte per item. It depends a little on your data and key, so assume 60+ bytes when on a 64 bit machine.

One can see it when looking at memcache’s code: https://github.com/memcached/memcached/blob/master/memcached.h

Here’s what makes up an item:

/**
 * Structure for storing items within memcached.
 */
typedef struct _stritem {
    struct _stritem *next;
    struct _stritem *prev;
    struct _stritem *h_next;    /* hash chain next */
    rel_time_t      time;       /* least recent access */
    rel_time_t      exptime;    /* expire time */
    int             nbytes;     /* size of data */
    unsigned short  refcount;
    uint8_t         nsuffix;    /* length of flags-and-length string */
    uint8_t         it_flags;   /* ITEM_* above */
    uint8_t         slabs_clsid;/* which slab class we're in */
    uint8_t         nkey;       /* key length, w/terminating null and padding */
    /* this odd type prevents type-punning issues when we do
     * the little shuffle to save space when not using CAS. */
    union {
        uint64_t cas;
        char end;
    } data[];
    /* if it_flags & ITEM_CAS we have 8 bytes CAS */
    /* then null-terminated key */
    /* then " flags length\r\n" (no terminating null) */
    /* then data with terminating \r\n (no terminating null; it's binary!) */
} item;

There are 3 pointers, which on a 64 bit machine sum up to 3 * 8 = 24 byte. There there a two time stamps, which should be 32 bit, so they make 8 byte in total. the following int should be 8 byte, the short should be 2, and u_int8 should be a single byte, so they sum up to 14.

This makes a total of 46 bytes.

If CAS is enable there is a 64 bit CAS value, which adds up 8 bytes: 54 Byte in total

Now it gets messy: Following is character data, where flags and length of data are printed as decimals (code can be found at https://github.com/memcached/memcached/blob/master/items.c, line 80). Given that flag is “0”, key is one char, and data is empty, this makes:

• 1 Byte key + 1 Byte 0
• 1 Byte “Space”, 1 Byte flag “0”, 1 Byte “Space”, 1 Byte “0”
• 2 Byte “\r\n” ( a line feed)
• 2 Byte for termination data

That’s another 10 Byte, making a total of 64 Byte for the smallest possible memcache item size (CAS enabled, you may disable it using the -C option).

If you get 65 Byte, maybe your client set a flag of “10” or such.