Question

I’m having issues in my code due to the multiple representations that a same float number can have.
For instance, these number are considered to be the same:

• 0.0299999400
• 0.0300000000

I don’t care much for a big precision, I need to calculate CRC of those numbers and they should be the same, so my approach was to use this code:

private static float Rounding(float v, float p)
{
    return Mathf.Round (v * p) / p; 
}

Where p is my precision.

That seems to be working well, but in that case, if I used p = 1e7, the first number multiplication would result in 299999.4 and rounded to 299999, becoming 0.0299999, while 0.03 would remain 0.03. That’s my problem.

Am I doing the right thing? Maybe the answer is that I can’t use a p this big, but I don’t know how floats work, so I don’t know the size of p to work in all situations.

However, I know that my float values resides in a range that is not as big as the float range (±1.5e−45 to ±3.4e38). What I mean is, if I know that my floats range is [-100; +100], I can assume that it will have enough decimal digits for p to be 3. Is that right?

Is there another way to do this?

Answer 1

You should convert the numbers to decimal and then round to the required precision. decimals are as accurate as they are precise, so you can then e.g. convert them to string and hash without having any such problems.