I have a test in my JUnit suite which warns me about a performance drop. The test works along these lines: Usually, I can run 100’000 operations per second, but the load on the build server can vary, so I’m happy with anything > 25’000.
Today, the build failed with 24’300 ops/s. When I ran the test with Java 7 (rc1, debug flavor), I got 5’000 ops/s. The former would be OK, the latter is a red flag that the test must catch.
Now the simple solution would be to reduce my goal to 20’000 but I was wondering if there was a better way to define the “safe range” of performance test if the machine can be under load.
Rather than measure wall-clock time, measure CPU time via the ThreadMXBean. If you’re looking to support 50k ops/second, then each operation should take < 20,000 nanoseconds.
I don’t know if the garbage collector threads will be accessible through this bean. If not, you will need to get their execution times via the GarbageCollectorMXBean