Question

See edits below

There is no casting going on with the termination check. I would think the < and the ++ would be as fast with ints and longs on a 64bit machine. But I guess not?

int: 65 milliseconds:

public void testWTF() throws Exception {
    int runs = 10;
    long hs = 0;
    long timeSum = 0;
    for (int run = 0; run < runs; run++) {
        int term = Integer.MAX_VALUE;
        long start = System.currentTimeMillis();
        // ***** loop to be tested ******
        for (int i = 0; i < term; i++) {
            hs++;
        }
        timeSum += (System.currentTimeMillis() - start);
        System.out.println("hs = " + hs);
        hs = 0;

    }
    System.out.println("timeSum = " + timeSum);
    System.out.println("avg time = " + (timeSum / runs) + " for " + runs + " runs");
    System.out.println("hs = " + hs);
}

long: 1445 milliseconds

public void testWTF() throws Exception {
    int runs = 10;
    long hs = 0;
    long timeSum = 0;
    for (int run = 0; run < runs; run++) {
        long term = Integer.MAX_VALUE;
        long start = System.currentTimeMillis();
        // ***** loop to be tested ******
        for (long i = 0; i < term; i++) {
            hs++;
        }
        timeSum += (System.currentTimeMillis() - start);
        System.out.println("hs = " + hs);
        hs = 0;

    }
    System.out.println("timeSum = " + timeSum);
    System.out.println("avg time = " + (timeSum / runs) + " for " + runs + " runs");
    System.out.println("hs = " + hs);
}

hardware: 64-bit Xeon running windows 7 64bit.

edit: I updated this to do several iterations. For 1 million runs with the int version, the average time is 65 milliseconds. The long version takes too long for 1 million, 1000 and even 100. For 10 runs the average time is 1447 milliseconds.

Also, I’m using hs outside the loop so that the loop does not get jitted away.

Answer 1

This is a very bad/unreliable/unrealistic way of doing benchmarks, since the JIT isn’t really given a chance to do much optimization — you only run the benchmarks once, and you measure the first run.

Basically, Java’s JIT will optimize your code significantly more once it sees your code getting used extensively. In a real program, the JIT will be optimizing any critical loops, so if you want a benchmark that mimics the real world, you have to convince the JIT to kick in.

The simplest way to get an accurate benchmark in Java is to use a tool like Caliper that knows how to properly warm up the JIT and get accurate measurements, and then see if the results are more consistent.