Question

My website has a very granular permissions system for every client, making it fairly large in size.

To keep a cap on database queries I have been loading bitmasks from a mysql database at the start of a users sessions and then saving it as session data, so it looked something like this. This has allowed me to one make one (albeit it complex JOIN query) query per user session without creating a huge session file.

"permissions" => array(
    "type 1" => 'bitfield'
    "type 2" => 'bitfield'
     "type 3" => array(
         entity id = 'bitfield'
         entity id = 'bitfield')
     "type 4" => array(
         entity id = 'bitfield'
         entity id = 'bitfield')
)

Permissions are entirely group based, so every person in a given group would have this replicated in their session data.

However bitmasks are starting to be a pain to use and I’m looking to move to using an acl. The reason I didn’t use an acl in the first place however was to minimise database usage..

So.. now I am going to have an entirely database/cache driven acl without any bitmasks. However storing huge arrays of permissions in user session data doesn’t seem ideal. (do you agree?)

I think the way to go is to use a flat file cache to store groups permissions. Would the easiest way to do this be a file per group? Would this change when there are 4,000 + groups each with 4 permission types (2 permission types are global with combined 40 or so permissions, 2 types are local permissions with combined 40 or so permissions per entity (each type has maybe 3 or 4 lots of 20 permission!). Edit: for clarity this means 160 – 200 permission entries per group

This seems like it would be a fairly huge cache! Would it be best to just have huge database usage on every page load? This kind of data size made bitmasks far easier but they are just simply not flexible enough anymore.

This is made harder by the fact files are served by 2 different servers (session stickied so saving bitfields to session data wasn’t a problem), so any cache would have to be synced between the servers. The db is on a seperate server connected by private network with supposedly 1gig connection.

Can any solutions be suggested? I think quick access cache such as memcached with this much data will just blow my memory usage out of the water? I am tempted to just user lots of database queries but I think that may put too heavy a strain on the db server.

Fairly large question, I hope its clear. If any of it needs clarification let me know. Any solutions will be greatly appreciated!

Chris

Answer 1

I don’t think that a data structure with ~40 entries in it stored in session is particularly huge. So really, what this might come down to design-wise is how to best get this information into the application with best performance.

If you are beginning to run up against performance problems and your infrastructure budget would allow for it, I would think your might look at migrating this solution to more of a service oriented architecture that could be shared across any number of servers. I am personally a huge believer in this sort of architecture as it can really help you in dealing with problems of scale.

You could expose “permissions” as a service that could be consumed by this application (or perhaps other applications you may need to develop in the future). It might look something like this:

• In-memory caching layer (memcached or similar) which is where application makes its initial call to look for permission information based on group. If the data does not exist here the next layer services the request
• RESTful API. You make a simple GET request for the permissions on your group after first having a cache miss. This will need to call to the database layer to get information. It would also do things like populating the cache on a cache miss, invalidating and repopulating the cache in cases where a client POSTs some new permissions data or PUTs update to an existing set of permissions.
  • DB layer this is accessed only by RESTful service. Perhaps MySQL perhaps a NoSQL technology if you have more complex non-relational data structures.

For your service you could probably have a very smal database server (because the database itself should be queried infrequently once the cache is populated). A memory caching server that has sufficient memory to meet your storage needs (or possibly a small cluster of servers if redundancy is needed), and a relatively small server to handle REST API (this should be infrequently accessed as well once cache is developed. The good thing is that they are several memcached or similar service out there that you can use relatively cheaply (like Amazon Elasticache). Really the in-memory cache would be taking the brunt of the traffic from the application servers so you would not need to scale up the REST server of DB server much at all as your traffic grows.

Hope this helps in your thought process a little.