Question

This question is specifically pertaining to Couchbase, but I believe it would apply to anything with the memcached api.

Lets, say I am creating a client/server chat application, and on my server, I am storing chat session information for each user in a data bucket. Well after the chat session is over, I will remove the session object from the data bucket, but at the same time I also want to persist it to a permanent NoSQL datastore for reporting and analytics purposes. I also want session objects to be persisted upon cache eviction, when sessions timeout, etc.

Is there some sort of “best practice” (or even a function of Couchbase that I am missing) that enables me to do this efficiently and maintaining best possible performance of my in memory caching system?

Answer 1

Using Couchbase Server 2.0, you could setup two buckets (or two separate clusters if you want to separate physical resources). On the session cluster, you’d store JSON documents (the value in the key/value pair), perhaps like the following:

{ 
  "sessionId" : "some-guid",
  "users" : [ "user1", "user2" ],
  "chatData" : [ "message1", "message2"],
  "isActive" : true,
  "timestamp" : [2012, 8, 6, 11, 57, 00]
}

You could then write a Map/Reduce view in the session database that gives you a list of all expired items (note the example below with the meta argument requires a recent build of Couchbase Server 2.0 – not the DP4.

function(doc, meta) {
  if (doc.sessionId && ! doc.isActive) {
    emit(meta.id, null);
  }
}

Then, using whichever Couchbase client library you prefer, you could have a task to query the view, get the items and move them into the analytics cluster (or bucket). So in C# this would look something like:

var view = sessionClient.GetView("sessions", "all_inactive");
foreach(var item in view) 
{
   var doc = sessionClient.Get(item.ItemId);
   analyticsClient.Store(StoreMode.Add, item.ItemId, doc);
   sessionClient.Remove(item.ItemId);
}

If you instead, wanted to use an explicit timestamp or expiry, your view could index based on the timestamp:

function(doc) {
   if (doc.sessionId && ! doc.isActive) {
      emit(timestamp, null);
   }
}

Your task could then query the view by including a startkey to return all documents that have not been touched in x days.

var view = sessionClient.GetView("sessions", "all_inactive").StartKey(new int[] { DateTime.Now.Year, DateTime.Now.Months, DateTime.Now.Days-1);

foreach(var item in view)
{
   var doc = sessionClient.Get(item.ItemId);
   analyticsClient.Store(StoreMode.Add, item.ItemId, doc);
   sessionClient.Remove(item.ItemId);
}

Checkout http://www.couchbase.com/couchbase-server/next for more info on Couchbase Server 2.0 and if you need any clarification on this approach, just let me know on this thread.

— John