equivalent to log10(2^24) ≈ 7.225 decimal digits Wikipedia Precision: 7 digits MSDN 6 std::numeric_limits<float>::digits10
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
If in doubt, read the spec. The C++ standard says that
digits10is:That’s a little vague; fortunately, there’s a footnote:
Those are defined in the C standard; let’s look it up there:
So
std::numeric_limits<float>::digits10is the number of decimal digits such that any floating-point number with that many digits is unchanged if you convert it to afloatand back to decimal.As you say, floats have about 7 digits of decimal precision, but the error in representation of both fixed-width decimals and floats is not uniformly logarithmic. The relative error in rounding a number of the form 1.xxx.. to a fixed number of decimal places is nearly ten times larger than the relative error of rounding 9.xxx.. to the same number of decimal places. Similarly, depending on where a value falls in a binade, the relative error in rounding it to 24 binary digits can vary by a factor of nearly two.
The upshot of this is that not all seven-digit decimals survive the round trip to float and back, but all six digit decimals do. Hence,
std::numeric_limits<float>::digits10is 6.There are not that many six and seven digit decimals with exponents in a valid range for the
floattype; you can pretty easily write a program to exhaustively test all of them if you’re still not convinced.