Question

This is not a new problem for me. From C to PERL to Python on Windows Mobile, Windows XP and other Windows versions this problem persists and f**ks my nerves.

Now in my latest script it again happens.
To be more concrete: I have coded in Python a trivial script. Now the script writes correctly to the file when run from the debugger, but out of the debugger it does not work correctly.
It does not write to file when it should.
I am using python 2.6 with eclipse and pydev.

This is the code

import httplib2
import thread

ht = httplib2.Http();
list = []
k = 0

def check(proxy, port):
    global list
    global k
    try:
        head = ht.request(proxy, 'HEAD')
    except:
        return
    k = k + 1
    list.append(proxy)
    list.append(port)


def OnListCaller(ProxyList, OutFile, NListLen):
    global list
    global k
    filei = open(ProxyList, 'r')
    fileo = open(OutFile, 'a')

    while 1:
        proxy = filei.readline()
        if not proxy: continue
        port = filei.readline()

        proxy = proxy.rstrip()
        port = port.rstrip()

        thread.start_new(check, (proxy, port,))

        if k >= NListLen:
            for t in list:
                fileo.write(t + "\n")
            list = []
            fileo.close()
            fileo = open(OutFile, 'a')
            k = 0


OnListCaller('C:\proxy\input.txt', 'C:\proxy\checked.txt', 1)   

The problem is in the OnListCaller at the if k>=NListLen.
The file should be updated when k is >= then a given value.
Thanks to everyone.

Answer 1

Regarding the code:
It looks like the actual problem is one relating to threads, not to files:

Whilst you are executing this code:

    for t in list:
        fileo.write(t + "\n")
    list = []
    fileo.close()
    fileo = open(OutFile, 'a')
    k = 0

list is being modified by the threads you have spawned. I don’t know the details of how ‘for x in y’ works with threads, but I imagine it is missing out the elements added to the list after the body for loop has been executed the first time.

To solve this you need a mutex for list which you lock for the entirety of this for loop (until you have cleared the list), and which you lock whenever you are adding an item to the list.