Question

I was reading about the MySQL: Using Connector/Net with Connection Pooling. It is suggested to not use a global MySqlConnection object and manually open/close it. Further, it’s suggested to use the MySqlHelper instead of working with MySqlCommand objects.

So, assuming I have a CONNECTION_STRING setup, I could do something like:

MySqlHelper.ExecuteNonQuery(CONNECTION_STRING,
    @"INSERT INTO `table1` (`col3`,`col4`) VALUES (@col3, @col4 );",
    (new List<MySqlParameter>() {
                new MySqlParameter("@col3", @"another value"),
                new MySqlParameter("@col4", @"some value")    
            }).ToArray()
    );

Now, imagine that I make the above call a few thousand times in async parallel tasks (via TPL).

There’s some max allowed connections on my MySQL server, and I start to get exceptions thrown claiming the user had exceeded max connections. I thought the connection pooling was suppose to handle this automatically for me? Isn’t the driver suppose to pool and queue them automatically for me?

I have 1500 max connections allowed currently, and I added this in my connection string:

Pooling=True;minimumPoolSize=0;maximumpoolsize=100;

Yet, I still get max connections exceeded. I need some advice with how to handle many async inserts without exploding this max connections limit.

Answer 1

In addition to max_connections, the max_user_connections is what is important here. It can be queried for:

show variables like 'max_user_connections';

To avoid exceeding max connections for your MySQL user you’ll need to make sure the maximumpoolsize (in the connection string) is set to something lower than max_user_connections and that other apps are not opening connections with that same user account whose total could exceed the cap. So for example, I could set mine to 5 given I have a cap of 15.

Capping this is very common by most hosting services to limit the resources you are using.

Performing a larger batch of queries in a single connection could also be a viable workaround.

In addition, sometimes hosts limit max_queries_per_hour and other such variables.