I have seen sudden spike in my application / platform memory utilization.In the GC verbose logs I have seen below :
1285.946: [GC 1285.946: [**ParNew (promotion failed)**: 353920K-353920K(353920K), 0.8003983 secs]1286.747: [CMS1287.338:
[CMS-con current-sweep: 7.902/9.624 secs] [Times: user=96.62 sys=2.35,
real=9.62 secs] (**concurrent mode failure**):
2531317K->1161025K(2752512K), 24.8330303 secs]
2860005K->1161025K(3106432K), [CMS Perm : 37117K->3 6905K(62368K)],
25.6341706 secs] [Times: user=26.41 sys=0.05, real=25.63 secs] [ POA RootPOA - rid: 17 oid: 00 17 2E 29 23 33 49 34 25 3E opname: ping -
process request ]
1312.367: [GC 1312.367: [ParNew: 314624K->30240K(353920K), 0.0188874 secs] 1475649K->1191266K(3106432K), 0.0194380 secs] [Time s: user=0.40
sys=0.00, real=0.02 secs]
1313.249: [GC 1313.249: [ParNew: 344864K->39296K(353920K), 0.0300220 secs] 1505890K->1201198K(3106432K), 0.0305488 secs]
ParNew (promotion failed ),concurrent mode failure :
I believe the sudden spike in the memory is visible because of GC failure. Explain and How to resolve this.
From the lines visible, I would assume that your application code creates too much garbage objects. The CMS failure Full GC was able to collect 1.4GB of garbage. So it is not a fragmentation issue, but an issue with the CMS not catching up.
The [CMS perm] makes me curious what your GC settings are. Perhaps you could give CMS more CPUs to run? Or you could enlarge the New Space to avoid premature promotion to old.
But chances are, that you run inefficient code. I would attach a profiler to it and look for garbage allocation hot spots.