Having two similar applications: one managed and the other unmanged. Both utilizing a heavy allocation pattern for large objects. I.e. they are requesting a lot of those objects in (long running) loop and both releasing those objects pretty right away after usage. The managed app uses IDisposable() which gets called immediately. The unmanaged is utilizing the destructors.
Some but not all of the objects could be reused. So, a pool is considered in order to increase execution speed and minimize the risk of memory fragmentation.
Which application would you expect to profit more from a pooling and why?
@Update: This is about a mathematical library. Therefore, those large objects will be arrays of value type. Mostly large enough for LOH. I am positive, pooling would improve performance on the managed side a lot. Many libraries exist – for managed/unmanaged environments. None of them I know does such pooling really. I am wondering why?
Feels strange, but I’ll try to answer it myself. May I’ll get some comments on it:
Both platforms suffer from fragmentation if very large objects are allocated and freed in a heavy manner (loops). In case of unmanaged apps, allocations are made from the virtual address space directly. Usually the working array will be wrapped in a class (c++), providing operator overloads for nice short syntax, some reference handling and a destructor, which makes sure, the array is freed immediately when running out of scope. Nevertheless, the arrays requested do not have the same size all the time – if larger arrays are requested, the same address block cannot get reused which may lead to fragmentation over time. Furthermore, there is no way of finding the block, which exactly serves the array length requested. The OS would simply use the first block, which is large enough – even if it was larger as needed and may could have been better fullfilling the request for an later upcoming even larger array. How could pooling improve that situation?
Imaginable would be, to use larger arrays for smaller requests. The class would handle the transition from the true length of the underlying array to the virtual length needed for the outside world. The pool could help to deliver the “first array, which is long enough” – in contrast to the OS, which will always give the exact length. This could possibly limit fragmentation, since less holes are created in virtual address space. On the other side, the overall memory size would increase. For nearly random allocation patterns, pooling would bring little to no profit, but only eat rare memory, i guess.
On the managed side, the situation is worse. First of all, two possible targets for fragmentation exist: virtual address space and the managed large object heap. Latter in that case would more be a collection of individual seqments, individually allocated from the OS. Each seqment would mostly be used for a single array only (since we are talking of really big arrays here). If one array is freed by the GC, the whole segment is returned to the OS. So fragmentation would not be an issue in the LOH (references: my own thoughts and some empirical observations using VMMap, so any comments very welcome!).
But since the LOH segments are allocated from the virtual address space, fragmentation is an issue here too – just like for unmanaged applications. In fact, the allocation pattern for both applications should look very similar for the memory manager of the OS. (?) With one distinction: the arrays are freed by the GC all at the same time. However, “really large arrays” will produce a lot of pressure on the GC. Only a relatively small number of “really large arrays” can be hold at the same time until collection occurs. Basically, the application usually spends a reasonable amount of time (seen about 5..45%) in the GC, also because virtually all collections will be expensive Gen2 collections and almost every allocation will result in such a Gen 2 collection.
Here pooling may help considerably. Once the arrays are not freed to the OS but rather collected in the pool, they are immediately available for further requests. (This is one reason, why IDisposable is not only meant for unmanaged resources). The framework/library does only have to make sure, the arrays are placed in the pool early enough and to enable the reuse of larger arrays for situations, where actually a smaller size is needed.