Question

First off, I’m not a math guy, so large number precision rarely filters into my daily work. Please be gentle. 😉

Using NumPy to generate a matrix with values equally divided from 1:

>>> m = numpy.matrix([(1.0 / 1000) for x in xrange(1000)]).T
>>> m
matrix[[ 0.001 ],
       [ 0.001 ],
       ...
       [ 0.001 ]])

On 64-bit Windows with Python 2.6, summing rarely works out to 1.0. math.fsum() does with this matrix, it doesn’t if I change the matrix to use smaller numbers.

>>> numpy.sum(m)
1.0000000000000007
>>> math.fsum(m)
1.0
>>> sum(m)
matrix([[ 1.]])
>>> float(sum(m))
1.0000000000000007

On 32-bit Linux (Ubuntu) with Python 2.6, summing always works out to 1.0.

>>> numpy.sum(m)
1.0
>>> math.fsum(m)
1.0
>>> sum(m)
matrix([[ 1.]])
>>> float(sum(m))
1.0000000000000007

I can add an epsilon to my code when assessing if the matrix sums to 1 (e.g. -epsilon < sum(m) < +epsilon) but I want to first understand what the cause of the difference is within Python, and if there’s a better way to determine the sum correctly.

My understanding is that the sum(s) are processing the machine representation of the numbers (floats) differently than how they’re displayed, and when sum’ing, the internal repesentation is used. Howeve,r looking at the 3 methods I used to calculate the sum it’s not clear why they’re all different, or the same between the platforms.

What’s the best way to correctly calculate the sum of a matrix?

If you’re looking for a more interesting matrix, this simple change will have smaller matrix numbers:

>>> m = numpy.matrix([(1.0 / 999) for x in xrange(999)]).T

Thanks in advance for any help!

Update
I think I figured something out. If I correct the value being stored to a 32-bit float the results match the 32-bit Linux sum’ing.

>>> m = numpy.matrix([(numpy.float32(1.0) / 1000) for x in xrange(1000)]).T
>>> m
matrix[[ 0.001 ],
       [ 0.001 ],
       ...
       [ 0.001 ]])
>>> numpy.sum(m)
1.0

This will set the matrix machine numbers to represent 32-bit floats, not 64-bit on my Windows test, and will sum correctly. Why is a 0.001 float not equal as a machine number on a 32-bit and 64-bit system? I would expect them to be different if I was trying to store very small numbers with lots of decimal places.

Does anyone have any thoughts on this? Should I explicitly switch to 32-bit floats in this case, or is there a 64-bit sum’ing method? Or am I back to adding an epsilon? Sorry if I sound dumb, I’m interested in opinions. Thanks!

Answer 1

It’s because you’re comparing 32-bit floats to 64-bit floats, as you’ve already found out.

If you specify a 32-bit or 64-bit dtype on both machines, you’ll see the same result.

Numpy’s default floating point dtype (the numerical type for a numpy array) is the same as the machine precision. This is why you’re seeing different results on different machines.

E.g.
The 32-bit version:

m = numpy.ones(1000, dtype=numpy.float32) / 1000
print repr(m.sum())

and the 64-bit version:

m = numpy.ones(1000, dtype=numpy.float64) / 1000
print repr(m.sum())

Will be different due to the differing precision, but you’ll see the same results on different machines. (However, the 64-bit operation will be much slower on a 32-bit machine)

If you just specify numpy.float, this will be either a float32 or a float64 depending on the machine’s native architecture.