Question

When performing many inserts into a database I would usually have code like this:

using (var connection = new SqlConnection(connStr))
{
  connection.Open();
  foreach (var item in items)
  {
     var cmd = new SqlCommand("INSERT ...")
     cmd.ExecuteNonQuery();
  }
}

I now want to shard the database and therefore need to choose the connection string based on the item being inserted. This would make my code run more like this

foreach (var item in items)
{
  connStr = GetConnectionString(item);
  using (var connection = new SqlConnection(connStr))
  {
    connection.Open();      
    var cmd = new SqlCommand("INSERT ...")
    cmd.ExecuteNonQuery();
  }
}

Which basically means it’s creating a new connection to the database for each item. Will this work or will recreating connections for each insert cause terrible overhead?

Answer 1

I assume you are talking about C#/.NET. In that case the connections are pooled by the framework, so the overhead of creating them that way isn’t that high.

Edit

As pointed out by @TomTom transactions should also be taken into considerations. If you are doing inserts into different databases on the same server you can use a normal SQL transaction for that. If the databases are on different servers you would need to use an MSDTC transaction for coordinating them across the database servers. Anyways the best way to handle transactions are by wrapping the relevant code in a TransactionScope. This doesn’t conflict with opening and closing (in reality reusing from a pool) database connections.

using(new TransactionScope())
{
    // Update code to various databases, opening and closing connections.
}

With SQL2005 or newer the TransactionScope will default to a SQL Transaction first and then automatically escalate it to an MSDTC transaction if needed.