Question

eps returns the distance from 1.0 to the next largest double-precision number, so I can use it to interpret the numbers value on negative weight position. But for very large number with value on high positive weight position, what can I use to interpret?

I mean that I need to have some reference to count out computation noise on numbers obtained on Matlab.

Answer 1

Have you read “What Every Computer Scientist Should Know About Floating-Point Arithmetic”?

It discusses rounding error (what you’re calling “computation noise”), the IEEE 754 standard for representation of floating-point numbers, and implementations of floating-point math on computers.

I believe that reading this paper would answer your question, or at least give you more insight into exactly how floating point math works.

---

Some clarifications to aid your understanding – too big to fit in the comments of @Richante’s post:

Firstly, the difference between realmin and eps:

• realmin is the smallest normalised floating point number. You can represent smaller numbers in denormalised form.
• eps is the smallest increment between distinct numbers. realmin = eps(realmin) * 2^52.

“Normalised” and “denormalised” floating point numbers are explained in the paper linked above.

Secondly, rounding error is no indicator of how much you can “trust” the nth digit of a number.

Take, for example, this:

>> ((0.1+0.1+0.1)^512)/(0.3^512)

ans =

    1.0000

We’re dividing 0.3^512 by itself, so the answer should be exactly one, right? We should be able to trust every digit up to eps(1).

The error in this calculation is actually 400 * eps:

>> ((0.1+0.1+0.1)^512)/(0.3^512) - 1

ans =

  9.4591e-014

>> ans / eps(1)

ans =

   426

The calculation error, i.e. the extent to which the nth digit is untrustworthy, is far greater than eps, the floating-point roundoff error in the representation of the answer. Note that we only did six floating-point operations here! You can easily rack up millions of FLOPs to produce one result.

I’ll say it one more time: eps() is not an indicator of the error in your calculation. Do not attempt to display : “My result is 1234.567 +/- eps(1234.567)”. That is meaningless and deceptive, because it implies your numbers are more precise than they actually are.

eps, the rounding error in the representation of your answer, is only 1 part per billion trillion or so. Your real enemy is the error that accumulates every time you do a floating point operation, and that is what you need to track for a meaningful estimate of the error.