Question

Imagine I write a simple calculator application, that just calculates simple stuff like

1.5 + 30 + 9755 – 30 – 20000 + 999900.54

I remember slightly that there were some precision problems when using floating point numbers. At which point would my calculator app start to create wrong results? Most of the time, I would just calculate integers like 1 + 2 – 963422, but sometimes I may enter a floating point number. I have no big clue about where the precision problems would start to take effect. just the very last numbers of that double? like -963419.0000000000003655? Or how would that look like? And any idea how to catch those?

Answer 1

Citing Wikipedia:

In addition to loss of significance,
inability to represent numbers such as
π and 0.1 exactly, and other slight
inaccuracies, the following phenomena
may occur:

• Cancellation: subtraction of nearly equal operands may cause extreme loss
  of accuracy. This is perhaps the most
  common and serious accuracy problem.
• Conversions to integer are not intuitive: converting (63.0/9.0) to
  integer yields 7, but converting
  (0.63/0.09) may yield 6. This is
  because conversions generally truncate
  rather than round. Floor and ceiling
  functions may produce answers which
  are off by one from the intuitively
  expected value.
• Limited exponent range: results might overflow yielding infinity, or
  underflow yielding a subnormal number
  or zero. In these cases precision will
  be lost.
• Testing for safe division is problematic: Checking that the divisor
  is not zero does not guarantee that a
  division will not overflow and yield
  infinity.
• Testing for equality is problematic. Two computational
  sequences that are mathematically
  equal may well produce different
  floating-point values. Programmers
  often perform comparisons within some
  tolerance (often a decimal constant,
  itself not accurately represented),
  but that doesn’t necessarily make the
  problem go away.

In order to avoid such problems, you need to analyze your specific calculations in order to minimize error propagation.