Question

I have a inline function does a frequency to period conversion. The calculation precision has to be using type long, not type double. Otherwise, it may cause some rounding errors. The function then converts the result back to double. I was wondering in below code, which line would keep the calculation in type long. No matter the parameter bar is 100, 100.0 or 33.3333.

double foo(long bar)
{
  return 1000000/bar;
  return 1000000.0/bar;
  return (long)1000000/bar;
  return (long)1000000.0/bar;
}

I tried it myself, and the 4th line works. But just wondering the concept of type conversion in this case.

EDIT:

One of the error is 1000000/37038 = 26, not 26.9993.

Answer 1

This problem, as you posed it, doesn’t make sense.

bar is of an integral type, so 1000000/bar will surely be less than 1000000, which can be represented exactly by a double¹, so there’s no way in which performing the calculation all in integral arithmetic can give better precision – actually, you will get integer division, that in this case is less precise for any value of bar, since it will truncate the decimal part. The only way you can have a problem in a long to double conversion here is in bar conversion to double, but if it exceeds the range of double the final result of the division will be 0, as it would be anyway in integer arithmetic.

Still:

1000000/bar

performs a division between longs: 1000000 is an int or a long, depending on the platform, bar is a long; the first operand gets promoted to a long if necessary and then an integer division is performed.

1000000.0/bar

performs a division between doubles: 1000000.0 is a double literal, so bar gets promoted to double before the division.

(long)1000000/bar

is equivalent to the first one: the cast has precedence over the division, and forces 1000000 (which is either a long or an int) to be a long; bar is a long, division between longs is performed.

(long)1000000.0/bar

is equivalent to the previous one: 1000000.0 is a double, but you cast it to a long and then integer division is performed.

---

1. The C standard, to which the C++ standard delegates the matter, asks for a minimum of 10 decimal digits for the mantissa of doubles (DBL_DIG) and at least 10**37 as representable power of ten before going out of range (DBL_MAX_10_EXP) (C99, annex E, ¶4).