Question

Preamble:

1. All data connection strings,
   connections, etc are created using
   DbProviderFactories.
2. Code is mixed C# and VB.Net from
   mulitple libraries.

I am mapping a DbDataReader to entities and have some benchmarks:

[0] retrieved 159180 records in 45135 ms
[1] retrieved 159180 records in 45008 ms
[2] retrieved 159180 records in 44814 ms
[3] retrieved 159180 records in 44987 ms
[4] retrieved 159180 records in 44914 ms
[5] retrieved 159180 records in 45224 ms
[6] retrieved 159180 records in 45829 ms
[7] retrieved 159180 records in 60762 ms
[8] retrieved 159180 records in 52128 ms
[9] retrieved 159180 records in 47982 ms  

This is a significant amount of time and extremely poor considering it only takes 17 seconds to query from Sql Server Management Studio. My select statement:

“SELECT * FROM tbl_MyTable”

Table contains 43 fields and probably isn’t indexed as best as it should; however, performing a select all, I wouldn’t expect indexing to be problematic. So … here is what I am doing:

Define an entity:

public class Concept
{
    #region Columns
    [DataParameter("ConceptID", DbType.Int32)]
    public Int32 ConceptID
    { get; set; }
    [DataParameter("ConceptName", DbType.String)]
    public string ConceptName
    { get; set; }
    [DataParameter("ConceptTypeID", DbType.Int32)]
    public Int32 ConceptTypeID
    { get; set; }
    [DataParameter("ActiveYN", DbType.Boolean)]
    public bool ActiveYN
    { get; set; }
    #endregion
}

Query DataReader:

for (int i = 0; i <= 99; i++)
{
    sw.Start();
    var results = session.QueryReader<Concept>(
        new SqlCommand(command), dr => new Concept());

    sw.Stop();

    Console.WriteLine("[{0}] retrieved {1} records in {2} ms", i, results.Count(), sw.ElapsedMilliseconds);
    sw.Reset();
}

… calling:

Public Function QueryReader(Of TEntity As {Class, New})(ByVal Command As DbCommand, _
                                                        ByVal Projection As Func(Of DbDataReader, TEntity)) _
                                                        As IEnumerable(Of TEntity)

    Dim list As IEnumerable(Of TEntity)

    Command.Connection = dataReader.NewConnection
    Command.Connection.Open()

    Using _reader As DbDataReader = Command.ExecuteReader()
        list = _reader.Query(Of TEntity)(Projection).ToList()
    End Using

    Command.Connection.Close()

    Return list
End Function

… and extension method QueryReader<T>: edit placement of new TEntity() – thanks @Henk

public static IEnumerable<TEntity> Query<TEntity>(this DbDataReader Reader,
    Func<DbDataReader, TEntity> Projection)
    where TEntity : class, new()
{
    //   moving this reflection to another class
    Dictionary<string, PropertyInfo> props;

    while (Reader.Read())
    {
        TEntity entity = new TEntity();

        if (!entities.TryGetValue(typeof(TEntity).ToString(), out props))
        {
            //  reflection over TEntity
            props = (from p in entity.GetType().GetProperties()
                     from a in p.GetCustomAttributes(typeof(DataParameterAttribute), false)
                     select p)
                     .ToDictionary(p => p.Name);

            entities.Add(typeof(TEntity).ToString(), props);
        }

        foreach (KeyValuePair<string, PropertyInfo> field in props)
        {
            if (null != Reader[field.Key] && Reader[field.Key] != DBNull.Value)
            { field.Value.SetValue(entity, Reader[field.Key], null); }
        }

        yield return entity;
    }
}

Any suggestions on increasing performance would be greatly appreciated …

---

Update

I implemented dapper-dot-net as @EtienneT suggested – here are the retrieval times:

[0] retrieved 159180 records in 6874 ms
[1] retrieved 159180 records in 6866 ms
[2] retrieved 159180 records in 6570 ms
[3] retrieved 159180 records in 6785 ms
[4] retrieved 159180 records in 6693 ms
[5] retrieved 159180 records in 6735 ms
[6] retrieved 159180 records in 6627 ms
[7] retrieved 159180 records in 6739 ms
[8] retrieved 159180 records in 6569 ms
[9] retrieved 159180 records in 6666 ms

Answer 1

Have you considered a micro ORM like dapper.net?

https://github.com/StackExchange/dapper-dot-net

It is made by the developers of StackOverflow and map an SQL query directly to your objects. It generates and caches IL code to map the SQL results to your objects. So the IL code is generated only one time per type. Never used this, but if you need performance to map your SQL results to .net objects, it is the library you need.