Question

When comparing the following code samples in python (the 300 is just an example; in practicality, assume there are 10,000,000 items that we need to loop through):

sum = 0
for i in xrange(0, 300):
    sum += i

Vs.

sum = 0
for i in xrange(0, 100):
    sum += i
for i in xrange(100, 200):
    sum += i
for i in xrange(200, 300):
    sum += i

Are both versions the same in terms of running time? I have a problem where I have a huge vector (say, 10,000 items). I don’t necessarily have to loop through all of it, unless some conditions are met (these conditions can only be assessed when the first loop — which using the first 100 items — is done).

Then, I should continue the looping only if some conditions are met (these conditions can only be assessed when the next 100 items are examined).

Then, I should continue the looping only if some conditions are met (these conditions can only be assessed when the next 100 items are examined).

Same story, until I reach the end of the vector…………….

Given the fact that I can solve my problem in a modular way, by examining points:

{0, 100}
{100, 200}
{200, 300}
……..
{9900, 10,000}

All I am trying is to get a sense whether the second approach is as efficient as the first one. Thanks.

note: the sum is used here for simplicity. In practice, the running time of the mathematical operations that will be used is significantly greater. That’s why I am trying to find ways to optimize the running time by examining different coding approaches…

Answer 1

What kind of improvement were you hoping to see?

timeit will help you observe the difference:

$ python -m timeit 'sum = 0
> for i in xrange(0, 300):
>     sum += i'
10000 loops, best of 3: 21.8 usec per loop

$ python -m timeit 'sum = 0
> for i in xrange(0, 100):
>     sum += i
> for i in xrange(100, 200):
>     sum += i
> for i in xrange(200, 300):
>     sum += i'
10000 loops, best of 3: 22.8 usec per loop

Personally, I would go for:

$ python -m timeit 'sum(xrange(300))'
100000 loops, best of 3: 5.34 usec per loop

This example is perhaps not the best way to assess whether you’ll be able to optimise your actual code, but it does demonstrate how you can test a small snippet for its running time.

Just remember what Donald Knuth said about optimisation 😉

Here’s a way you could checkpoint your iteration and break out if required:

>>> for index, item in enumerate(vect):
...     results = do_stuff_with(item)
...     if index % 100 == 0 and should_we_stop(results):
...         break

What would be even better is if you could have do_stuff_with() return a tuple indicating whether it has finished, then you can check every iteration whether you’re done, rather than wait:

>>> for item in vect:
...     finished, results = do_stuff_with(item)
...     if finished:
...         break

Just to reiterate (no pun intended) it’s very hard to say how (or even whether) to optimise your actual code without actually seeing it!