Question

We are preparing to scale the API side of an API-heavy web application. My (technically savvy) client proposes a rather unconventional approach to this: instead of balancing the load to several app servers, which would talk to a sharded database, he wants us to:

• “shard the app servers”, putting both app server code and db on each physical server, so that the app server only connects to its own db shard;
• have the app servers talk to each other when they need to access other shards (instead of talking to another shard’s DB directly);
• have the API client pick an app shard itself (on the client side, based on some stable hash) and talk directly to it.

The underlying reasoning is that this is the most natural thing to do it, and that this would allow us to move to a multisite distributed system in the future.

(The stack is PHP + Node.js on MySQL, although at this point a transition to MongoDB is considered too.)

Now, I don’t see huge problems with it off the shelf. It might get somewhat cumbersome to code these server-to-server interactions, but then it will surely have its own benefits. Basically I’m at a loss on whether this is a good idea or not.

What pros and cons come to your mind? I’m looking for technical issues and advantages here. Thanks!

Answer 1

This is just plain bad for many reasons.

• The API client should not know which app shard to talk to. This will limit you in ways you probably can’t foresee now, but may/will become a problem in the future. The API client should play dumb so you can route requests appropriately if an app server dies, changes, gets sharded again etc.
• What happens if your app code or database architecture is slow? (Not both at the same time, just one). Now you have a db shard slowing down an app shard.
• Your db+app shards will need to keep both app code+memory and db code+memory in RAM. This means the CPUs will spend more time swapping code and memory in and out to perform both sets of tasks.
• I’m finding it hard to put down in words, but this type of architecture screams ‘bad coupling’ and ‘no separation of concerns’ (probably not the right terminology but I hope you understand what I mean). You are putting two distinctly different types of applications (app server and database) onto one box. The management nightmare of updating them and routing around failed instances will be very difficult.

I hate to argue my point this way, but a lot of very smart people have dealt with these problems before and I’ve never heard of this type of architecture. There’s probably a reason for it. Not to mention there’s a lot of technology and resources out there that can help you handle traditional sharding and load balancing of app and database servers. If you go with your client’s suggested architecture you’re on your own.