Question

In my application I’m going to use floating point values to store geographical coordinates (latitude and longitude).

I know that the integer part of these values will be in range [-90, 90] and [-180, 180] respectively. Also I have requirement to enforce some fixed precision on these values (for now it is 0.00001 but can be changed later).

After studying single precision floating point type (float) I can see that it is just a little bit small to contain my values. That’s because 180 * 10^5 is greater than 2^24 (size of the significand of float) but less than 2^25.

So I have to use double. But the problem is that I’m going to store huge amounts of this values, so I don’t want to waste bytes, storing unnecessary precision.

So how can I perform some sort of compression when converting my double value (with fixed integer part range and specified precision X) to byte array in java? So for example if I use precision from my example (0.00001) I end up with 5 bytes for each value.
I’m looking for a lightweight algorithm or solution so that it doesn’t imply huge calculations.

Answer 1

To store a number x to a fixed precision of (for instance) 0.00001, just store the integer closest to 100000 * x. (By the way, this requires 26 bits, not 25, because you need to store negative numbers too.)