Question

I was just testing a quantizer class i have made for use in a project, and to simulate a relatively finely grained range of numbers (for the class to quantize into discrete steps), I made a For…Next loop using a Double as the incrementing value. Like so:

For d As Double = 0 To 1 Step 0.01
    ' logic here
Next

The results were not what I expected from my quantizer, and I scratched my head for a good while trying to figure out where my quantizing logic was wrong. In the end, I simply made a dump of the numbers produced by the loop, almost just for the hell of it. To my great surprise, the above loop didn’t produce a range of numbers in precise steps of 0.01..!

This is the loop I used:

For d As Double = 0 To 1 Step 0.01
    Me.TextBox1.AppendText(d.ToString & vbCrLf)
Next

And this is the list of numbers it printed:

0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
0.15
0.16
0.17
0.18
0.19
0.2
0.21
0.22
0.23
0.24
0.25
0.26
0.27
0.28
0.29
0.3
0.31
0.32
0.33
0.34
0.35
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
0.51
0.52
0.53
0.54
0.55
0.56
0.57
0.58
0.59
0.6
0.61
0.62
0.63
0.64
0.65
0.66
0.67
0.68
0.69
0.7
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78
0.79
0.8
0.810000000000001
0.820000000000001
0.830000000000001
0.840000000000001
0.850000000000001
0.860000000000001
0.870000000000001
0.880000000000001
0.890000000000001
0.900000000000001
0.910000000000001
0.920000000000001
0.930000000000001
0.940000000000001
0.950000000000001
0.960000000000001
0.970000000000001
0.980000000000001
0.990000000000001

Obviously, up until 0.8, it’s what I expected, but after that.. What is going on? Increasing the loop’s upper limit from 1 to, say, 3 will display similar bands of imprecise numbers at certain intervals. Do I have to rewrite the loop to something like this?

Dim d As Double
For i As Integer = 0 To 100
    d = i / 100
    ' logic here
Next

Answer 1

Floating point numbers (as doubles are)are approximations. The approximation your system uses for 0.8 plus the approximation your system uses for 0.01 comes closer to 0.810000000000001 than to 0.81. If you would like this to be exact, like you suggested yourself, use integers

For i As Int = 0 To 100
  Dim d as Double = i/100.0
' logic here
Next