Question

I just recently came across the Kahan (or compensated) summation algorithm for minimizing roundoff, and I’d like to know if there are equivalent algorithms for division and/or multiplication, as well as subtraction (if there happens to be one, I know about associativity). Implementation examples in any language, pseudo-code or links would be great!

Thanks

Answer 1

Subtraction is usually handled via the Kahan method.

For multiplication, there are algorithms to convert a product of two floating-point numbers into a sum of two floating-point numbers without rounding, at which point you can use Kahan summation or some other method, depending on what you need to do next with the product.

If you have FMA (fused multiply-add) available, this can easily be accomplished as follows:

p = a*b;
r = fma(a,b,-p);

After these two operations, if no overflow or underflow occurs, p + r is exactly equal to a * b without rounding. This can also be accomplished without FMA, but it is rather more difficult. If you’re interested in these algorithms, you might start by downloading the crlibm documentation, which details several of them.

Division… well, division is best avoided. Division is slow, and compensated division is even slower. You can do it, but it’s brutally hard without FMA, and non-trivial with it. Better to design your algorithms to avoid it as much as possible.

Note that all of this becomes a losing battle pretty quickly. There’s a very narrow band of situations where these tricks are beneficial–for anything more complicated, it’s much better to just use a wider-precision floating point library like mpfr. Unless you’re an expert in the field (or want to become one), it’s usually best to just learn to use such a library.