Question

I have 2 sets of data from different systems. About 20,000 records a piece. One set of data is a List<Objects>, the other is a RecordSet from a system I have no control over (I can only ask for records and receive the recordset).

I need to merge data between the 2, matching on a key that exists in both lists, by putting some data from the Recordset into the matching object structure. The nested loop I’ve tried is entirely too slow.

Is there a fast way to get the match and move on to the next object?

EDIT: EXISTING CODE

This is a very basic nested loop

results = _sr.SearchLst(ut.ToString(), searchSettings);

foreach (BL.Packet ePacket in eList) {

    for (Int32 j = 0; j < results.Rows.Length; j++) {

        String acckey = results.Rows[0].Data.GetValue(1).ToString();
        String taskname = results.Rows[0].Data.GetValue(2).ToString();

        if (acckey == ePacket.RecKey.ToString()) {
            ePacket.prop1 = taskname;
            ePacket.prop2 = acckey;
        }
    }
}

I did also look at Linq, but couldn’t get a grasp of how the two would go together…I’m not sure Linq would be faster anyway.

SOLUTION:

Here’s basically what I ended up doing.

//move object from list to dictionary
var dict = eList.Cast<BaseObj>().ToDictionary(o => o.RecKey, o => o);

results = _sr.SearchLst(ut.ToString(), searchSettings);
if (results.Rows.Length > 0) {

    //loop through all rows in recordset
    for (Int32 j = 0; j < results.Rows.Length; j++) {
        id = Convert.ToInt32(results.Rows[j].Data.GetValue(1)); 
        taskname = results.Rows[j].Data.GetValue(2).ToString();

        if (dict.ContainsKey(id)) {
            //recordset id found in dictionary, so grab taskname
            ePacket = ((BL.Packet)dict[id]);
            ePacket.prop1 = taskname;
        }
    }
}

//move dictionary back to list
List<BaseObj> eListReturn = new List<BaseObj>(dict.Values);
return eListReturn;

Thanks to everyone for the assistance!

Answer 1

Use a Dictionary<TypeOfKey, SomeObject> on the list collection you have in memory, populate the dictionary using the key as key and the object as value.

Loop over the record set. Using the dictionary you can do an O(1) lookup of the matching element, so you can eliminate the looping over the in-memory collection.