Question

I am beginning to add a memcached layer to my application. so far so good.

I realised quite quickly however that I will be needing to invalidate/delete large batches of keys in one go when someone uploads a file to the site in order to keep the data relevant.

I have done some reading and the most common way of getting around this problem seems to be setting a version number on each key, then rather than deleting the keys when a user uploads (because there could be so many permutations) you incrememnt the version number, inducing a cache miss next time the data is accessed.

I have no idea where to begin in order to get this coded and i’m not quite sure I totally get my head around it anyway.

My code currently looks like this:-

$memcache = new Memcache;


$memcache->connect('localhost', 11211) or die ("Could not connect");


$key = md5("songsearch_$textSearch.$rangeSearch");

The two variables in the key var above are retrieved from the get request which in turn retrieves a large amount of JSON. (Think a product catalogue).

These variables also determine the query itself which is dynamically put together dependant upon these variables. Ultimately all of this gives me a key that is unique to every individual query, even though from the same script I could have hundreds of keys being generated.

I hope the above is clear, if not please ask me to clarify any points in order to better help you answer my question.

Obviously later, to set the cache the results I am using this:-

$memcache->set($key, $output, MEMCACHE_COMPRESSED, time() + 24*60*60*365);

just before I encode the JSON.

So my question really is… How can I add some form of incremental versioning to this so that if a user uploads a file I can invalidate the whole lot of keys that have been generated by this script?

Huge thanks to anyone who gets me at least on the right track.

Answer 1

You are obviously on the right track. The only thing you’re missing: Store a version number in the memcache, which you retrieve before you build your key.

$memcache = new Memcache;
$memcache->connect('localhost', 11211) or die ("Could not connect");

// fetch the current version number
$version = $memcache->get('version_number');

// and add the version to the key
$key = md5("songsearch_$textSearch.$rangeSearch") . 'v' . $version;

So now, whenever somebody uploads new content, you simply increment the version number:

$memcache->increment('version_number');

This automatically leads to all existing queries to run into invalid entries.

To simplify the access, I recommend you wrap this in a new Memcache handler class (untested):

class VersionedMemcache extends Memcache {

    const VERSION_KEY = 'version_number';

    private $version = 1;

    public function __construct() {
        $version = parent :: get(self :: VERSION_KEY);
        if ($version === false) {
            $version = 1;
            $this->set(self :: VERSION_KEY, $version);
        }
        $this->version = $version;
    }

    public function get($key, &$flags = null) {
        $key = $key . '_v' . $this->version;
        return parent :: get($key, $flags);
    }

    public function incVersion() {
        $this->version = $this->increment(self :: VERSION_KEY);
        return $this->version;
    }

}

So now, you would simply amend your script to:

$memcache = new VersionedMemcache();

$memcache->connect('localhost', 11211) or die ("Could not connect");

$key = md5("songsearch_$textSearch.$rangeSearch");

// this will now result in a fetch on 'ab23...232afe_v1' instead of 'ab23...232afe'
$result = $memcache->get($key);

// when an upload happens, simply
$memcache->incVersion();