Question

For ultra-fast code it essential that we keep locality of reference- keep as much of the data which is closely used together, in CPU cache:

http://en.wikipedia.org/wiki/Locality_of_reference

What techniques are to achieve this? Could people give examples?

I interested in Java and C/C++ examples. Interesting to know of ways people use to stop lots of cache swapping.

Greetings

Answer 1

This is probably too generic to have clear answer. The approaches in C or C++ compared to Java will differ quite a bit (the way the language lays out objects differ).

The basic would be, keep data that will be access in close loops together. If your loop operates on type T, and it has members m1…mN, but only m1…m4 are used in the critical path, consider breaking T into T1 that contains m1…m4 and T2 that contains m4…mN. You might want to add to T1 a pointer that refers to T2. Try to avoid objects that are unaligned with respect to cache boundaries (very platform dependent).

Use contiguous containers (plain old array in C, vector in C++) and try to manage the iterations to go up or down, but not randomly jumping all over the container. Linked Lists are killers for locality, two consecutive nodes in a list might be at completely different random locations.

Object containers (and generics) in Java are also a killer, while in a Vector the references are contiguous, the actual objects are not (there is an extra level of indirection). In Java there are a lot of extra variables (if you new two objects one right after the other, the objects will probably end up being in almost contiguous memory locations, even though there will be some extra information (usually two or three pointers) of Object management data in between. GC will move objects around, but hopefully won’t make things much worse than it was before it run.

If you are focusing in Java, create compact data structures, if you have an object that has a position, and that is to be accessed in a tight loop, consider holding an x and y primitive types inside your object rather than creating a Point and holding a reference to it. Reference types need to be newed, and that means a different allocation, an extra indirection and less locality.