Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
When using excel if I calculate 6/100 to get the percentage value it returns 0.060000
but when i do the same calc in objective-c I get 0.059999998658895493
Does anyone know why there is a difference?
Floating point values cannot represent all fractional numbers exactly. This the same behavior we see with some numbers in decimal, 1/3 can not be represented exactly in decimal, it is only approximated as 0.33333333…
Computer fractional numbers are base 2 where our decimal numbers are base 10. Different fractional numbers can be represented with complete accuracy in different bases.
Because of this there is a number class in Cocoa,
NSDecimalNumberthat operates in base 10 and can represent 6/100 exactly as 0.060000The difference you see in Excel could be either because it is using decimal math (probably not) or that it is rounding the number so that the display inaccuracy is not apparent.