I have a problem dealing with bad_alloc. It is thrown during a std::vector.push_back when it tries to reloc and allocate 2Mb
The heaps status is
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
04e00000 00000002 67704 41088 67704 40735 135 135 0 25 LFH
External fragmentation 99 % (135 free blocks)
Virtual address fragmentation 39 % (135 uncommited ranges)
...
0c8f0000 00001002 82216 81384 82216 8247 172196 13 5 3 LFH
...
0: Heap 04e00000
Flags 00000002 - HEAP_GROWABLE
Reserved memory in segments 67704 (k)
Commited memory in segments 41088 (k)
Virtual bytes (correction for large UCR) 58644 (k)
Free space 40735 (k) (135 blocks)
External fragmentation 99% (135 free blocks)
Virtual address fragmentation 29% (135 uncommited ranges)
Virtual blocks 0 - total 0 KBytes
Lock contention 37
Segments 1
Low fragmentation heap 04e044f0
Lock contention 0
Metadata usage 0 bytes
Statistics:
Segments created 0
Segments deleted 0
Segments reused 0
Block cache:
Buckets info:
Size Blocks Seg Empty Aff Distribution
------------------------------------------------
------------------------------------------------
Default heap Front heap Unused bytes
Range (bytes) Busy Free Busy Free Total Average
------------------------------------------------------------------
0 - 1024 217 0 0 0 487 2
1024 - 2048 1 0 0 0 1 1
3072 - 4096 0 1 0 0 0 0
15360 - 16384 1 0 0 0 1 1
203776 - 204800 0 2 0 0 0 0
220160 - 221184 0 2 0 0 0 0
224256 - 225280 0 1 0 0 0 0
236544 - 237568 0 2 0 0 0 0
240640 - 241664 0 1 0 0 0 0
257024 - 258048 0 1 0 0 0 0
265216 - 266240 0 4 0 0 0 0
273408 - 274432 0 1 0 0 0 0
277504 - 278528 0 2 0 0 0 0
281600 - 282624 0 2 0 0 0 0
285696 - 286720 0 6 0 0 0 0
289792 - 290816 0 4 0 0 0 0
293888 - 294912 0 12 0 0 0 0
297984 - 299008 0 10 0 0 0 0
302080 - 303104 0 13 0 0 0 0
306176 - 307200 0 14 0 0 0 0
310272 - 311296 0 19 0 0 0 0
314368 - 315392 0 15 0 0 0 0
318464 - 319488 0 3 0 0 0 0
326656 - 327680 0 1 0 0 0 0
330752 - 331776 0 3 0 0 0 0
333824 - 334848 1 0 0 0 8 8
351232 - 352256 0 1 0 0 0 0
363520 - 364544 0 2 0 0 0 0
367616 - 368640 0 1 0 0 0 0
396288 - 397312 0 3 0 0 0 0
416768 - 417792 0 1 0 0 0 0
420864 - 421888 0 2 0 0 0 0
424960 - 425984 0 1 0 0 0 0
433152 - 434176 0 1 0 0 0 0
437248 - 438272 0 1 0 0 0 0
466944 - 467968 0 1 0 0 0 0
470016 - 471040 0 1 0 0 0 0
482304 - 483328 0 1 0 0 0 0
------------------------------------------------------------------
Total 220 135 0 0 497 2
0: Heap 0c8f0000
Flags 00001002 - HEAP_GROWABLE
Reserved memory in segments 82216 (k)
Commited memory in segments 81384 (k)
Virtual bytes (correction for large UCR) 81620 (k)
Free space 8247 (k) (172196 blocks)
External fragmentation 10% (172196 free blocks)
Virtual address fragmentation 0% (13 uncommited ranges)
Virtual blocks 5 - total 0 KBytes
Lock contention 3
Segments 1
Low fragmentation heap 43230048
Lock contention 0
Metadata usage 4096 bytes
Statistics:
Segments created 120
Segments deleted 2
Segments reused 0
Block cache:
5: 4096 bytes ( 54, 0)
6: 8192 bytes ( 26, 0)
7: 16384 bytes ( 13, 0)
8: 32768 bytes ( 8, 0)
9: 65536 bytes ( 9, 0)
10: 131072 bytes ( 5, 0)
11: 262144 bytes ( 4, 1)
Buckets info:
Size Blocks Seg Empty Aff Distribution
------------------------------------------------
64 25483 53 0 0 (53-25483)
72 2991 30 0 0 (30-2991)
80 750 15 0 0 (15-750)
88 46 1 1 0 (1-46)
96 42 1 1 0 (1-42)
104 39 1 1 0 (1-39)
112 36 1 1 0 (1-36)
120 33 1 1 0 (1-33)
128 63 1 1 0 (1-63)
136 60 1 1 0 (1-60)
144 56 1 1 0 (1-56)
152 53 1 1 0 (1-53)
160 51 1 1 0 (1-51)
168 48 1 1 0 (1-48)
200 40 1 1 0 (1-40)
208 39 1 1 0 (1-39)
224 36 1 1 0 (1-36)
280 29 1 1 0 (1-29)
312 26 1 1 0 (1-26)
7432 17 1 1 0 (1-17)
9736 26 1 1 0 (1-26)
10760 24 1 1 0 (1-24)
11784 22 1 1 0 (1-22)
------------------------------------------------
Default heap Front heap Unused bytes
Range (bytes) Busy Free Busy Free Total Average
------------------------------------------------------------------
0 - 1024 619437 172194 28854 1067 11637848 17
1024 - 2048 10 1 0 0 113 11
3072 - 4096 2 0 0 0 32 16
4096 - 5120 1 0 0 0 16 16
5120 - 6144 1 0 0 0 20 20
7168 - 8192 1 0 0 17 20 20
9216 - 10240 0 0 0 26 0 0
10240 - 11264 0 0 0 24 0 0
11264 - 12288 0 0 0 22 0 0
12288 - 13312 2 0 0 0 32 16
15360 - 16384 1 0 0 0 1 1
16384 - 17408 2 0 0 0 32 16
20480 - 21504 1 0 0 0 8 8
21504 - 22528 1 0 0 0 20 20
40960 - 41984 1 0 0 0 8 8
43008 - 44032 1 0 0 0 16 16
48128 - 49152 1 0 0 0 16 16
54272 - 55296 1 0 0 0 8 8
58368 - 59392 1 0 0 0 17 17
72704 - 73728 1 0 0 0 20 20
83968 - 84992 0 1 0 0 0 0
96256 - 97280 1 0 0 0 16 16
108544 - 109568 1 0 0 0 16 16
116736 - 117760 1 0 0 0 21 21
139264 - 140288 1 0 0 0 16 16
145408 - 146432 1 0 0 0 16 16
159744 - 160768 1 0 0 0 22 22
163840 - 164864 1 0 0 0 20 20
193536 - 194560 1 0 0 0 20 20
262144 - 263168 3 0 0 0 40 13
327680 - 328704 1 0 0 0 16 16
333824 - 334848 1 0 0 0 8 8
436224 - 437248 1 0 0 0 16 16
------------------------------------------------------------------
Total 619479 172196 28854 1156 11638454 17
My question is what External fragmentation 99% means and why is “heap fragmentation” and not “Virtual Address space fragmentation” ?
What is the difference between “External fragmentation” (99%) and “Virtual address fragmentation” (29%) ?
Creating a new heap, may help ?
PS: I have in the same process a LOT of managed code. The Private Working Set is 1,8Gb.
Basically you have a large number of very small object allocations, which cause the fragmentation. In addition you are mixing native and managed code in the same process and you are making big memory allocations. These approaches require memory management. It would be best to create a dedicated heap for large memory allocations on process startup to avoid this kind of problem. The heap should only be used for large memory allocations of native code.
In addition it would be a good idea to analyze why the managed code needs so much memory.