Question

My web application need to support scalability at both Web and DB tier.
I have the following components:

1. N Web servers (Tomcat)
2. M DB servers sharded by username as the shard key (PostgreSQL)

Our shard strategy as following:

1. The shard strategy is lookup table based, we have one index table (username,shardId), the other shard table (shardId,connectionString,loading).

2. We will monitor the shard DB periodically, and update the loading status field.

3. When create new user, we always pickup the shard with lowest loading, and store to index table.

4. The DB shards will be added or removed dynamically.

I have to implement an API like getDBConnection(username) to get one JDBC connection according to the shard key (in this case the login user).

The questions are :

1.How could I implement this API in a way that will work with a connection pool? Suppose each shard support 500 connections, how could i did this via Java code?

Answer 1

I probably wouldn’t connection pool in the application at all, I’d probably just use server-side connection pooling with pgpool-II or pgbouncer.

If I was to use in-application pooling I’d create per-shard connection pools and just pick a connection out of the appropriate pool for the shard. This will work with any connection pool implementation that lets you create pools programmatically, rather than declaratively. Each pool should try to close inactive connections quite aggressively. It looks like org.apache.tomcat.jdbc.pool.DataSource is suitable for this; see the Tomcat JDBC pool docs.

Since this will result in potentially large numbers of connections sitting around and lots of connects/disconnects, it will be very important to also run a server side connection pool to limit and share connections to each shard. Use something like pgbouncer or pgpool-II in transaction pooling mode on each shard to share a relatively small number of real PostgreSQL connections out among the larger number of connections from web workers.

I’d say you’ll want a pattern like this:

web1 [pool1]-----------------------------[server1-pgbouncer]------[server1-pg]
     [pool2]-------------------------------v         ^
     [pool3]---------v        [server2-pgbouncer]-----------------[server2-pg]
                     |                     ^         |
                    [server3-pgbouncer]---------------------------[server3-pg]
                     |                     |         |
web2 [pool3]---------^                     |         |
     [pool2]-------------------------------|         |
     [pool1]------------------------------------------

where each app pool connects to a pgbouncer on the shard server corresponding to that pool, and only that shard server’s pgbouncer ever talks to the shard’s PostgreSQL instance.