Question

I’ve been fiddling around with some sorting algorithms and timing them to see just how effective they are. To that end, I’ve created a static class containing a number of sorting algorithms for integers, and another class for timing them and exporting data to csv.

I’ve started looking at said data, and I’ve noticed an interesting trend. My program creates 5 different random arrays for testing, and it take the average of 10 different trials on each array for every sorting algorithm. The strange thing is that the first array seems to have a significantly longer average time for a few algorithms, but not for the others. Here’s some example data to back it up:
Dataset 1, Dataset 2, and Dataset 3 (times are in nanoseconds).

I’m not sure whether it has to do with certain algorithms, the way I implemented the algorithms, the JVM, or some other combination of factors. Does anybody know how this type of data happens?

Also, source code for all of this is available here. Look under the ‘src’ folder.

Answer 1

This looks like the effects of just-in-time compilation are present.

To reduce start-up time, when code is first run, it is directly interpreted from the byte code. Only after any particular piece of code has been run several times, the JIT decides it is a critical section and it’s worth re-compiling to native code. Then it replaces the method by its compiled form.

The effects of just-in-time compilation is that the start up time is extremely reduced (the entire application does not need to be compiled before it’s run) but also the critical sections run as fast as they can (because they eventually are compiled). It looks like the critical section got compiled somewhere within the first evaluation.

I am not sure why some algorithms do not exhibit the speedup from compilation. Most likely it’s because they share their critical section with some other previous algorithm. For example, the Insertion sort relies heavily on the swap method which got compiled during the selection sort evaluation. Another possible reason lies in noting that mergeSort, which runs consistently, does not rely heavily on function calls, so it does not benefit from inlining. Heap sort, on the other hand relies heavily on the siftDown method, which might be difficult for the compiler to inline.

If you mark your swap as final, it enables the compiler to inline the calls to this method. Since this method is small and it’s called often, inlining it (which is a part of the just in time compilation) will help the performance significantly.

To provide consistent test environment, you could disable just-in-time compilation while running the tests.