Question

I have a large array with around 20k objects in it. Each object has child objects in a big complicated tree structure with arrays in there too. Right now the app is developed using just a simple myObjectType[] myArray and it takes 13 seconds just to get the count of items in the array.

Is there a better type or is there a better way that I should be managing the array? 99% of the usage of the array is reading from it, but it currently takes almost 3 minutes to populate it.

EDIT:: add more info.

The app currently is loading all of this data into the giant array and then using the array as a data base. It then filters the data down based upon your selections from some drop down boxes and returns a subset to a datagrid to display. I don’t have the option to rewrite the whole thing to just pass the filters to the actual db…

EDIT: more info, sorry for the delay, was pulled into a meeting.

[Serializable]
public class PartsList : System.Collections.CollectionBase
{
  public virtual Part[] parts {get { return (Part[])List; } }
  public new virtual int Count { get{ return this.List.Count;}}

  public virtual CountryList GetCountries()
  {
    CountryList countries = new CountryList;
    //Code removed - goes through every sub item and makes a list of unique countries... 
    // Yes, could have been done better. 
    Return countries;
  }

}

/////////////////////////////////////

[Serializable]
public class Part
{
  private int id, blah, myvariable;
  private CountryList countries;  //formatted the same way as this class...
  private YearList  years; 
  private ModelList models;
  private PriceHistoryList priceHistoryList;
  // there are a couple more like these...
}

This is why it takes 3minutes to load.
– 20k parts
– 1-2 countries per part
– 1-10 years per part
– 1-12 models per part
– 1-10 price history per part

When I stop the debugger on this line:
PartsList mylist = new PartsList;
//populate the list here
if (list.Count != 0 ) <- the debugger takes 13 seconds to get off this line after hitting f10. doing a “quick watch” on list just gives me an error for the counts value.

What I’m really looking for is, is there a better variable type to replace the array’s with since they are nested internally…

UPDATE Jan 29 2010
Did some searching and it appears that due to the object designs, it is lazy loading one object at a time into memory, causing a TON of sql calls to be fired. Also the Count seems to take so long because a combo of using CollectionBase and complex objects where it is retrieving each object, counting it then going to the next. Plan now IS to move app to 2008 (.net 3.5 from 1.1) and do a rewrite of the back end of the application so that it does not pre-load 350mb into memory…

Thanks everyone for their inputs.

Answer 1

20’000 objects (e.g. references in fact) is peanuts. The count will return instantenously. If you have trouble, it’s not because of the array class.

Which collection to use in the end depends on what you want to do with it.

Before optimizing, always make sure to find the bottleneck. Often this is not what one expects first, and therefore you should absolutely use a profiler to see what is actually taking up so much time.