Question

I went upon a very strange behavior (to me) :

    int generate_scenario_one_pass(FILE *out, double freq_mhz) {
        unsigned int d_freq, d_freq_test;
        d_freq              = (int)(freq_mhz * 20);
        d_freq_test         = (int)(float)(freq_mhz * 20);
        printf("when freq_mhz = %.1f, d_freq = 0x%04X, d_freq_test = 0x%04X\n", freq_mhz, d_freq, d_freq_test);
    }

The whole code is not here, but it’s not relevant. This function is called several times with increasing values, starting from 2110.0 with an increment of 0.1.

when freq_mhz = 2110.0, d_freq = 0xA4D8, d_freq_test = 0xA4D8
when freq_mhz = 2110.1, d_freq = 0xA4DA, d_freq_test = 0xA4DA
when freq_mhz = 2110.2, d_freq = 0xA4DC, d_freq_test = 0xA4DC
when freq_mhz = 2110.3, d_freq = 0xA4DD, d_freq_test = 0xA4DE

At the last iteration, d_freq is wrong! But d_freq_test has the correct value. So my issue was solved by casting from double to float, then from float to int. I wanted to know why.

This was compiled using MSVC++ 6.0 on a x86 CPU.

Answer 1

There are many numbers that cannot be represented exactly as a floating-point number – and 0.1 is among them (it will be rounded to the closest number that can be represented – something along the lines of 0.0999999999999998). When using double, 2110.3 happens to be represented by a number that is slightly smaller than 2110.3, thus giving the “wrong” result when you multiply by 20 and cast to int (which will round down), while 2110.3 as a float will be represented by a number that is slightly bigger than 2110.3, thus giving the expected outcome.