Question

In attempting to remove all filenames not starting with a particular substring from a list, I’ve run into the following unexpected behavior:

>>> allfiles = os.listdir(mydir)
>>> allfiles
['dwcpybyext.sh', 'dwlaunch', 'libupdate.sh', 'ntpsync.sh']
>>> for f in allfiles:
...    if f.startswith('n') == False:
...        allfiles.remove(f)
...
>>> allfiles
['dwlaunch', 'ntpsync.sh']

This theoretically should have removed every filename not starting with 'n' from the list. Instead, it’s left one starting with 'd' in the list. If I change the loop to use if f.startswith('d') == False: I get ['dwcpybyext.sh', 'dwlaunch', 'ntpsync.sh'] — the last item doesn’t even contain a 'd' character.

Why am I seeing this behavior? It seems unlikely to be a bug in Python’s list.remove() method — I get the same behavior if I substitute del allfiles[allfiles.index(f)], and .remove() is basically just an alias for that anyway.

Answer 1

It is a really bad idea to modify list while iterating it. Try the next:

allfiles = filter(lambda x: x.startswith('n'), allfiles)