Question

I have an iterator of numbers, for example a file object:

f = open("datafile.dat")

now I want to compute:

mean = get_mean(f)
sigma = get_sigma(f, mean)

What is the best implementation? Suppose that the file is big and I would like to avoid to read it twice.

Answer 1

If you want to iterate once, you can write your sum function:

def mysum(l):
    s2 = 0
    s = 0
    for e in l:
        s += e
        s2 += e * e
    return (s, s2)

and use the result in your sigma function.

Edit: now you can calculate the variance like this: (s2 – (s*s) / N) / N

By taking account of @Adam Bowen’s comment,
keep in mind that if we use mathematical tricks and transform the original formulas
we may degrade the results.