Question

I have written a simple SUDOKU solver. To roughly test the performance I’m using simple System.currentTimeMillis calls.

I have prepared a set of initial sudoku configuration in text file. The program reads the file and solves each sudoku configuration. When running the tests I have noticed that the first 3-4 solve runs are really slower than the rest and by slower I mean by order of magnitude.

There is sample pseudo-code snippet:

main(){
     while(file has lines){
         configuration = readLine();
         Solver s = new Solver(configuration);

         now1 = System.currentTimeMillis();
         s.solve();
         now2 = System.currentTimeMillis();

         System.out.print(now2 - now1);
     }
}

I measure only solve() method, so IO is not a problem, I even hardcoded some data into program – still first few slower. The difficulty of puzzle is not an issue as well I have tried different permutations and difficulties of configuration and always the same – first few are slower.

My question is – why is that and is there a way to prevent it?

Answer 1

This is supposed to happen. The JIT compiler optimizes code that gets called more often as your program runs for longer.

This only reflects the general fact that the technique you’re using to test performance simply isn’t reliable in Java.