I’ve been fighting decimal precision in C# coming from a SQL Decimal (38,30) and I’ve finally made it all the way to a rounding oddity. I know I’m probably overlooking the obvious here, but I need a little insight.
The problem I’m having is that C# doesn’t produce what I would consider to be consistent output.
decimal a = 0.387518769125m;
decimal b = 0.3875187691250002636113061835m;
Console.WriteLine(Math.Round(a, 11));
Console.WriteLine(Math.Round(b, 11));
Console.WriteLine(Math.Round(a, 11) == Math.Round(b, 11));
Yields
0.38751876912
0.38751876913
False
Uhh, 0.38751876913? Really? What am I missing here?
From MSDN:
If the digit in the decimals position is odd, it is changed to an even digit. Otherwise, it is left unchanged.
Why am I seeing inconsistent results? The additional precision isn’t changing the ‘digit in the decimals position’…
From MSDN:
In your first case
there is a single digit to the right of the 11th place, and that number is
5. Therefore, since position 11 is even, it is left unchanged. Thus, you getIn your second case
there is not a single digit to the right of the 11th place. Therefore, this is straight up grade-school rounding; you round up if the next digit is greater than 4, otherwise you round down. Since the digit to the right of the 11th place is more than 4 (it’s a 5), we round up so you see
You’re not. The results are completely consistent with the documentation.