Question

I’m creating a mobile app and it requires a API service backend to get/put information for each user. I’ll be developing the web service on ServiceStack, but was wondering about the storage. I love the idea of a fast in-memory caching system like Redis, but I have a few questions:

1. I created a sample schema of what my data store should look like. Does this seems like it’s a good case for using Redis as opposed to a MySQL DB or something like that?

   schema http://www.miles3.com/uploads/redis.png

2. How difficult is the setup for persisting the Redis store to disk or is it kind of built-in when you do writes to the store? (I’m a newbie on this NoSQL stuff)

3. I currently have my setup on AWS using a Linux micro instance (because it’s free for a year). I know many factors go into this answer, but in general will this be enough for my web service and Redis? Since Redis is in-memory will that be enough? I guess if my mobile app skyrockets (hey, we can dream right?) then I’ll start hitting the ceiling of the instance.

Answer 1

What to think about when desigining a NoSQL Redis application

1) To develop correctly in Redis you should be thinking more about how you would structure the relationships in your C# program i.e. with the C# collection classes rather than a Relational Model meant for an RDBMS. The better mindset would be to think more about data storage like a Document database rather than RDBMS tables. Essentially everything gets blobbed in Redis via a key (index) so you just need to work out what your primary entities are (i.e. aggregate roots)
which would get kept in its own ‘key namespace’ or whether it’s non-primary entity, i.e. simply metadata which should just get persisted with its parent entity.

Examples of Redis as a primary Data Store

Here is a good article that walks through creating a simple blogging application using Redis:

http://www.servicestack.net/docs/redis-client/designing-nosql-database

You can also look at the source code of RedisStackOverflow for another real world example using Redis.

Basically you would need to store and fetch the items of each type separately.

var redisUsers = redis.As<User>();
var user = redisUsers.GetById(1);
var userIsWatching = redisUsers.GetRelatedEntities<Watching>(user.Id);

The way you store relationship between entities is making use of Redis’s Sets, e.g: you can store the Users/Watchers relationship conceptually with:

SET["ids:User>Watcher:{UserId}"] = [{watcherId1},{watcherId2},...]

Redis is schema-less and idempotent

Storing ids into redis sets is idempotent i.e. you can add watcherId1 to the same set multiple times and it will only ever have one occurrence of it. This is nice because it means you don’t ever need to check the existence of the relationship and can freely keep adding related ids like they’ve never existed.

Related: writing or reading to a Redis collection (e.g. List) that does not exist is the same as writing to an empty collection, i.e. A list gets created on-the-fly when you add an item to a list whilst accessing a non-existent list will simply return 0 results. This is a friction-free and productivity win since you don’t have to define your schemas up front in order to use them. Although should you need to Redis provides the EXISTS operation to determine whether a key exists or a TYPE operation so you can determine its type.

Create your relationships/indexes on your writes

One thing to remember is because there are no implicit indexes in Redis, you will generally need to setup your indexes/relationships needed for reading yourself during your writes. Basically you need to think about all your query requirements up front and ensure you set up the necessary relationships at write time. The above RedisStackOverflow source code is a good example that shows this.

Note: the ServiceStack.Redis C# provider assumes you have a unique field called Id that is its primary key. You can configure it to use a different field with the ModelConfig.Id() config mapping.

Redis Persistance

2) Redis supports 2 types persistence modes out-of-the-box RDB and Append Only File (AOF). RDB writes routine snapshots whilst the Append Only File acts like a transaction journal recording all the changes in-between snapshots – I recommend adding both until your comfortable with what each does and what your application needs. You can read all Redis persistence at http://redis.io/topics/persistence.

Note Redis also supports trivial replication you can read more about at: http://redis.io/topics/replication

Redis loves RAM

3) Since Redis operates predominantly in memory the most important resource is that you have enough RAM to hold your entire dataset in memory + a buffer for when it snapshots to disk. Redis is very efficient so even a small AWS instance will be able to handle a lot of load – what you want to look for is having enough RAM.

Visualizing your data with the Redis Admin UI

Finally if you’re using the ServiceStack C# Redis Client I recommend installing the Redis Admin UI which provides a nice visual view of your entities. You can see a live demo of it at:
http://servicestack.net/RedisAdminUI/AjaxClient/