Question

I play with a Java benchmark (SPECJbb) but observed some performance degradation with large pages (2M per page) enabled. Our server is NUMA arch based, has 80 cores (160 logical cpu when HT enabled) and 260G memory.

The benchmark consumes about 200G memory, so I allocated 200G large in kernel large page pool.But the performance is 6%~8% decline. I think I should have some performance gain because this benchmark used 200G memory.

My question is about, in what situation, using large page will degrade performace?
Our OS is RHEL6.3 and I tried HotSpot JDK 1.6.38 and 1.6.37, both these two versions I found such degradation. Thanks.

Answer 1

Large pages should usually bring up to 5% performance improvement.

You should check whether the large pages were properly allocated:

• see cat /proc/meminfo and check for actual large pages allocation size
• How big is your Heap? Did you use the -XX:+UseLargePages option? See that the JVM managed to use large pages, there will be a warning in the JVM output log if there was any problem

Also, there could be a problem with LargePages + NUMA architecture. How much memory there is for each core? See:

• Using Large Pages, they report about large pages performance degradation of 6-8% due NUMA
  setting.
• Huge Pages and NUMA issues on Linux