Question

When rendering in HLSL using SurfaceFormat.Rgba64 I only have a range of [0, 1]. To work around this I wish to use the entire (not sure of the correct terminology?) range of numbers after the floating point.

I can do this by using two constants to shift the numbers to the left (16BIT_MULTIPLIER) and right (16BIT_INVMULTIPLIER)

16BIT_MULTIPLIER = 1000
16BIT_INVMULTIPLIER = 1 / 16BIT_MULTIPLIER

What is the maximum amount I can shift the numbers to the right and maintain correct precision? In other words what is the maximum value 16BIT_MULTIPLIER can be?

Answer 1

In a standard 32-bit IEEE 754 floating-point value, the maximum ratio between the most significant bit in a number and its least significant bit is 2²³. This is because the floating-point encoding has 24 bits (including an “implicit bit”) for the significand (or fraction). So the highest bit may be 2²³ while the lowest bit is 2⁰ (1), or the highest bit may be 2¹² while the lowest bit is 2^-11, for examples. In your case, you may wish to use a highest bit of 2^-1 with a lowest bit of 2^-24. You tell us the range you are mapping to ([0, 1]) but not the range you are mapping from, so we cannot tell you the scaling factor to use.

If you are mapping from unsigned 16-bit numbers, with a range of [0, 65536), you could use a scaling factor of 1/65536. (In many languages, you would write 1./65536 to get a floating-point constant; 1/65536 would be an integer expression that evaluates to zero.) This scaling would map all your numbers into the target range with available precision “below” the numbers, but it would not leave any margin for adding numbers or other operations that increase magnitude. If you want to leave room for more arithmetic while keeping results in the interval [0, 1], then you need a larger ratio (smaller scaling factor).

You should use powers of two for the scaling factor. Multiplying by powers of two has no rounding error in binary floating point. Multiplying by powers of ten or their inverses (which are necessarily approximate since inverses of powers of ten cannot be exactly represented in binary floating point) usually cause rounding errors.

In a standard 64-bit IEEE floating-point value, the maximum ratio is 2⁵².

You ask what the maximum value 16BIT_MULTIPLIER can be, but it is unlikely this is what you actually want. You can make 16BIT_MULTIPLIER as large as you like until the low bits of the numbers reach 2^-149. At that point, you reach the end of the exponent range (2^-126 for the high bit), and bits with smaller values cannot be represented (in 32-bit floating point).