Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
If I have variables a, b, an c of type double, let c:=a/b, and give a and b values of 7 and 10, then c’s value of 0.7 registers as being LESS THAN 0.70.
On the other hand, if the variables are all type extended, then c’s value of 0.7 does not register as being less than 0.70.
This seems strange. What information am I missing?
There is no representation for the mathematical number
0.7in binary floating-point. Your statement computes incthe closestdouble, which (according to what you say, I didn’t check) is a little below 0.7.Apparently in extended precision the closest floating-point number to 0.7 is a little above it. But there still is no exact representation for 0.7. There isn’t any at any precision in binary floating-point.
As a rule of thumb, any non-integer number whose last non-zero decimal is not 5 cannot be represented exactly as a binary floating-point number (the converse is not true: 0.05 cannot be represented exactly either).