Question

When finding the inverse square root of a double, is it better to clamp invalid non-positive inputs at 0.0 or MIN_DBL? (In my example below double b may end up being negative due to floating point rounding errors and because the laws of physics are slightly slightly fudged in the game.)

Both division by 0.0 and MIN_DBL produce the same outcome in the game because 1/0.0 and 1/DBL_MIN are effectively infinity. My intuition says MIN_DBL is the better choice, but would there be any case for using 0.0? Like perhaps sqrt(0.0), 1/0.0 and multiplication by 1.#INF000000000000 execute faster because they are special cases.

double b = 1 - v.length_squared()/(c*c);

#ifdef CLAMP_BY_0
if (b < 0.0) b = 0.0;
#endif

#ifdef CLAMP_BY_DBL_MIN
if (b <= 0.0) b = DBL_MIN;
#endif

double lorentz_factor = 1/sqrt(b);

double division in MSVC:

1/0.0     = 1.#INF000000000000
1/DBL_MIN = 4.4942328371557898e+307

Answer 1

When dealing with floating point math, “infinity” and “effectively infinity” are quite different. Once a number stops being finite, it tends to stay that way. So while the value of lorentz_factor is “effectively” the same for both methods, depending on how you use that value, later computations can be radically different. sqrt(lorentz_factor) for instance remains infinite if you clamp to 0, but will actually be calculated if you clamp to some very very small number.

So the answer will largely depend on what you plan on doing with that value once you’ve clamped it.