Question

What’s the best and fastest way to access a value from the previous iteration in a for loop, assuming that the object will be very large (example, a cursor object which has 100,000+ records)

Using a simple example:

tmp = [
         ['xyz', 335], ['zzz', 338], ['yyy', 339], ['yyy', 442], 
         ['abc', 443], ['efg', 444], ['ttt', 446], ['fff', 447]
      ]

for x in tmp:
   if not prev:
     prev = x[1]
   print 'seq: ', x[1], 'prev seq:', prev, 'variance: ', x[1]-prev
   prev = x[1]

Is this the most optimal way to handle this?

Based on the responses below i did some testing:
tmp was created with 500 lists, the average of running it 20 times is shown below.

results:

Mines: 0,623
Dave snippet1: 0,605
Dave snippet2: 0,586
Catchmeifyoutry (edited code): 0,707

Answer 1

Your code is going to be doing the “if not prev” test every time round the loop, even though it only applies to the first element.
Also your code seems broken to me – the first time round the loop the prev and current values are the same.

I would do it like this, assuming that there is at least one element:

tmp_iter = iter(tmp)
prev = tmp_iter.next()

for x in tmp_iter: 
   print 'seq: ', x[1], 'prev seq:', prev[1], 'variance: ', x[1]-prev[1]
   prev = x

this could be optimised further by getting rid of the indexing:

tmp_iter = iter(tmp)
[_, prev] = tmp_iter.next()

for [_, x] in tmp_iter: 
   print 'seq: ', x, 'prev seq:', prev, 'variance: ', x-prev
   prev = x

I use the assignment to spit the list into its constituent parts, and assign the first element to _ because it is not used.