Question

I’ve tested row cache, and find out, that it requires large amount of Heap – I would like to verify this theory.

This is my test key space:
{
TestCF: {

    row_key_1: {
        { clientKey: "MyTestCluientKey" },
        { tokenSecret: "kd94hf93k423kf44" },
        { verifier: "hfdp7dh39dks9884" },
        { callbackUrl: "http%3A%2F%2Fprinter.test.com%2Fready" },
        { accountId: "234567876545"},
        { mytestResourceId: "ADB112"},
        { dataTimestamp: "1308903420400" },
        { dataType: "ACCESS_PERMANENT"}
    },
    row_key_2: {
        { clientKey: "MyTestCluientKey" },
        { tokenSecret: "qdqergvhetyhvetyh" },
        { verifier: "wtrgvebyjnrnuiucewrqxcc" },
        { callbackUrl: "http%3A%2F%2Fprinter.test.com%2Fready" },
        { accountId: "23456789746534"},
        { mytestResourceId: "DQERGCWRTHB"},
        { dataTimestamp: "130890342333200" },
        { dataType: "ACCESS_LIMITED"}
    },

…
row_key_x: {
….
},

}

}

Each row in CF: TestCF contains 8 columns. Row cache is enabled, key cache is disabled. Row hit rate 0.99 – this is read only test.

My test loads 1.500.000 rows into cache – and this allocates about 3.5GB heap – this is about 2KB pro single row – this is a lot….

Is it possible, that single row (8 columns) can allocate about 2KB heap?

Thank you,
Maciej

Answer 1

Yes, rule of thumb is that a row in a memtable or row cache will take about 8x to 12x its serialized size. This is one reason the off-heap cache is the default in 1.0 if you have JNA installed: off-heap stores the rows in serialized form.