Question

I’m developing for a platform without a math library, so I need to build my own tools. My current way of getting the fraction is to convert the float to fixed point (multiply with (float)0xFFFF, cast to int), get only the lower part (mask with 0xFFFF) and convert it back to a float again.

However, the imprecision is killing me. I’m using my Frac() and InvFrac() functions to draw an anti-aliased line. Using modf I get a perfectly smooth line. With my own method pixels start jumping around due to precision loss.

This is my code:

const float fp_amount = (float)(0xFFFF);
const float fp_amount_inv = 1.f / fp_amount;

inline float Frac(float a_X)
{
    return ((int)(a_X * fp_amount) & 0xFFFF) * fp_amount_inv;
}

inline float Frac(float a_X)
{
    return (0xFFFF - (int)(a_X * fp_amount) & 0xFFFF) * fp_amount_inv;
}

Thanks in advance!

Answer 1

If I understand your question correctly, you just want the part after the decimal right? You don’t need it actually in a fraction (integer numerator and denominator)?

So we have some number, say 3.14159 and we want to end up with just 0.14159. Assuming our number is stored in float f;, we can do this:

f = f-(long)f;

Which, if we insert our number, works like this:

0.14159 = 3.14159 - 3;

What this does is remove the whole number portion of the float leaving only the decimal portion. When you convert the float to a long, it drops the decimal portion. Then when you subtract that from your original float, you’re left with only the decimal portion. We need to use a long here because of the size of the float type (8 bytes on most systems). An integer (only 4 bytes on many systems) isn’t necessarily large enough to cover the same range of numbers as a float, but a long should be.