Question

I hope I framed the question right. I am trying to force myself to be a better programmer. By better I mean efficient. I want to write a program to identify the files in a directory and read each file for further processing. After some shuffling I got to this:

for file in os.listdir(dir):
    y=open(dir+'\\'+file,'r').readlines()
    for line in y:
        pass
    y.close()

It should be no surprise that I get an AttributeError since y is a list. I didn’t think about that when I wrote the snippet.

I am thinking about this and am afraid that I have five open files (there are five files in the directory specified by dir.

I can fix the code so it runs and I explicitly close the files after opening them. I am curious if I need to or if Python handles closing the file in the next iteration of the loop. If so then I only need to write:

for file in os.listdir(dir):
    y=open(dir+'\\'+file,'r').readlines()
    for line in y:
        pass

I am guessing that it(python) does handle this effortlessly. The reason I think that this might be handled is that I have changed the object/thing that y is referencing. When I start the second iteration there are no more memory references to the file that was opened and read using the readlines method.

Answer 1

Python will close open files when they get garbage-collected, so generally you can forget about it — particularly when reading.

That said, if you want to close explicitely, you could do this:

for file in os.listdir(dir):
    f = open(dir+'\\'+file,'r')
    y = f.readlines()
    for line in y:
        pass
    f.close()

However, we can immediately improve this, because in python you can iterate over file-like objects directly:

for file in os.listdir(dir):
    y = open(dir+'\\'+file,'r')
    for line in y:
        pass
    y.close()

Finally, in recent python, there is the ‘with‘ statement:

for file in os.listdir(dir):
    with open(dir+'\\'+file,'r') as y:
        for line in y:
            pass

When the with block ends, python will close the file for you and clean it up.

(you also might want to look into os.path for more pythonic tools for manipulating file names and directories)