Question

Can anyone explain this weirdness:

Dim result as single = 0
result = CType("8.01", Single) * 100 ' result=801.0 as expected
result = CType("8.02", Single) * 100 ' result=802.000061 --- not expected

Further to the above

result = 8.02 * 100 ' result = 802.0 as expected

Answer 1

Single (or float) only has seven significant digits, so everything that might be printed beyond those is pretty much bogus anyway. This is a normal artifact of floating-point arithmetic. You can only rely on up to 4 digits after the decimal point with your number (as three significant digits are already before the decimal point).

Further elaborated, the numbers 8.01 and 8.02 can’t be represented exactly using binary storage (as neither 0.01 nor 0.02 can be represented exactly with sums of fractions of the form 1/2ⁿ). The details may vary from number to number but you may see extraneous digits outside the normal precision range when using such numbers.

This doesn’t affect the fact that 801 and 802 can be represented exactly but you have no exact number to start with in this case.

ETA: In fact, what you see when you include the calculation directly is just this: The compiler will evaluate the calculation for you and simply write 802 into the program. You can use Reflector to verify that. Also floating point literals in the source code are likely Double by default so you have much greater precision to begin with here. If result is a Single this will be downcasted to Single and the error in the 16th digit after the decimal point is simply thrown away since it can’t fit into Single anyway.