Question

We’re migrating SQL to Azure. Our DAL is Entity Framework 4.x based. We’re wanting to use the Transient Fault Handling Block to add retry logic for SQL Azure.

Overall, we’re looking for the best 80/20 rule (or maybe more of a 95/5 but you get the point) – we’re not looking to spend weeks refactoring/rewriting code (there’s a LOT of it). I’m fine re-implementing our DAL’s framework but not all of the code written and generated against it anymore than we have to since this is already here only to address a minority case. Mitigation >>> elimination of this edge case for us.

Looking at the possible options explained here at MSDN, it seems Case #3 there is the “quickest” to implement, but only at first glance. Upon pondering this solution a bit, it struck me that we might have problems with connection management since this circumvent’s Entity Framework’s built-in processes for managing connections (i.e. always closing them). It seems to me that the “solution” is to make sure 100% of our Contexts that we instantiate use Using blocks, but with our architecture, this would be difficult.

So my question: Going with Case #3 from that link, are hanging connections a problem or is there some magic somewhere that’s going on that I don’t know about?

Answer 1

I’ve done some experimenting and it turns out that this brings us back to the old “managing connections” situation we’re used to from the past, only this time the connections are abstracted away from us a bit and we must now “manage Contexts” similarly.

Let’s say we have the following OnContextCreated implementation:

private void OnContextCreated()
{
    const int maxRetries = 4;
    const int initialDelayInMilliseconds = 100;
    const int maxDelayInMilliseconds = 5000;
    const int deltaBackoffInMilliseconds = initialDelayInMilliseconds;

    var policy = new RetryPolicy<SqlAzureTransientErrorDetectionStrategy>(maxRetries,
                                                                            TimeSpan.FromMilliseconds(initialDelayInMilliseconds),
                                                                            TimeSpan.FromMilliseconds(maxDelayInMilliseconds),
                                                                            TimeSpan.FromMilliseconds(deltaBackoffInMilliseconds));
    policy.ExecuteAction(() =>
            {
                try
                {
                    Connection.Open();
                    var storeConnection = (SqlConnection) ((EntityConnection) Connection).StoreConnection;
                    new SqlCommand("declare @i int", storeConnection).ExecuteNonQuery();
                    //Connection.Close();
                    // throw new ApplicationException("Test only");
                }
                catch (Exception e)
                {
                    Connection.Close();

                    Trace.TraceWarning("Attempted to open connection but failed: " + e.Message);

                    throw;
                }
            }
        );
}

In this scenario, we forcibly open the Connection (which was the goal here). Because of this, the Context keeps it open across many calls. Because of that, we must tell the Context when to close the connection. Our primary mechanism for doing that is calling the Dispose method on the Context. So if we just allow garbage collection to clean up our contexts, then we allow connections to remain hanging open.

I tested this by toggling the comments on the Connection.Close() in the try block and running a bunch of unit tests against our database. Without calling Close, we jumped up to ~275-300 active connections (from SQL Server’s perspective). By calling Close, that number hovered at ~12. I then reproduced with a small number of unit tests both with and without a using block for the Context and reproduced the same result (different numbers – I forget what they were).

I was using the following query to count my connections:

SELECT s.session_id, s.login_name, e.connection_id,
      s.last_request_end_time, s.cpu_time, 
      e.connect_time
FROM sys.dm_exec_sessions AS s
INNER JOIN sys.dm_exec_connections AS e
ON s.session_id = e.session_id
WHERE login_name='myuser'
ORDER BY s.login_name

Conclusion: If you call Connection.Open() with this work-around to enable the Transient Fault Handling Block, then you MUST use using blocks for all contexts you work with, otherwise you will have problems (that with SQL Azure, will cause your database to be “throttled” and ultimately taken offline for hours!).