I have an application that is pretty memory hungry. It holds a large amount of data in some big arrays.
I have recently been noticing the occasional OutOfMemoryException. These OutOfMemoryExceptions are occurring long before my application (ASP.Net) has used up the 800mb available to it.
I have track the issue down to the area of code where the array is resized. The array contains a structure that is 74bytes in size. (I know that you shouldn’t create struct’s that are bigger than 16bytes), but this application is a port from a Vb6 application). I have tried changing the struct to a class and this appears to have fixed the problem for now.
I think the reason that changing to a class solves the problem has to do with the fact that when using a struct and the array is resized, a segment of memory that is large enough to store the new array needs to be reserved (e.g. (currentArraySize + increaseBySize)*74) cannot be found. This leads to the OutOfMemoryException.
This isn’t the case with a class as each element of the array only needs 8bytes to store a pointer to the new object.
Is my thinking correct here?
When you resize an array it will create a new one to hold the new data, then copy over the data, and you will have two copies of the same data in memory at the same time. Just as you expected.
When using structs the array will occupy the
struct size * number of elements. When using a class it will only contain the pointer.The same scenario is also true for List which increase in size over time, thus it’s smart to initialize it with the expected number of items to avoid resizing and copying.
On 32bit systems you will hit outofmem around ~800mb, as you are aware of. One solution you can try is to but your structs on disk and read them when needed. Since they are a fixed size you can easily jump into the correct position at the file.
I have a project on Codeplex for handling large amounts of data. It has a type of Array with possibility for autogrowing, which might help your scenario if you run into problems with keeping it all in memory again.