Question

I have a very weird problem in the most simple piece of VB.NET code:

Public Sub ReadIDs(i_oDataReader As Data.SqlClient.SqlDataReader)

    m_aFullIDList = New Generic.List(Of Integer)

    While i_oDataReader.Read
        If i_oDataReader.IsDBNull(0) = False Then
            m_aFullIDList.Add(i_oDataReader.GetInt32(0))
        End If
    End While

    m_iTotalNumberOfRecords = m_aFullIDList.Count

End Sub

Basically the procedure accepts a DataReader as a parameter, loops thru all the records in the reader, adding them to a Generic list of integer.

When SQL Stored Procedure that is behind the reader returns relatively small amount of records (below 100K) reader reads all the records and the final count matches the resultset. But when the number of records is over 1 million – results become unpredictable.

E.g. When I know that SP rerturns 1.5M of records – final count of records in Generic list could be 200K, 500K etc.

Any idea what is causing it and how to fix it? Thanks!

Answer 1

It may help to use the ExecuteReader overload that allows you to specify a CommandBehavior, and pass CommandBehavior.SequentialAccess as recommended in this MSDN topic:

Retrieving Large Data (ADO.NET).