Question

I typed abs(sin(pi)) into MATLAB. I expected it to give me the absolute value of pi, hoping to mitigate the natural rounding error with ‘abs’ (absolute value). I still get a rounding error. What should I do instead and why am I wrong?

Answer 1

Floating point numbers are always going to give you problems like this. That’s why it’s common to write:

if (x - TestValue < 0.000001)

instead of

if (x == TestValue)

I would recommend trying to round the value down a decimal place or two, using something like this:

x = floor(x * 1e15) / 1e15;

which rounds x down to the nearest 1e15. You could also use round or ceil. This article has some more information on similar strategies for rounding.