I’ve got a question about floating point precision. As I understand it, ULP is the distance between two consecutive discrete, finite numbers in a given specification. Or the unit in the least significant place as the name states. In any case, I saw this on the Java spec for sinh in java.lang.Math:
“The computed result must be within 2.5 ulps of the exact result.”
I don’t understand how you can have a ULP error that isn’t a whole number. If you’re off by 2.5 ULPs, doesn’t that imply that there’s a place less significant than the least significant place? What does it mean to have an error of 2.5 ULPs?
Consider the calculation 3/2. The exact result is 1.5. In integer arithmetic, the result is 1. That’s an error of 0.5, i.e. half a ULP.
Similar reasoning may be applied to floating-point arithmetic.