Question

I have one main directory called Forfun. Inside this directory, there are 7-8 sub-directorates as Test1, Test2, Test3 and so on. Note there are some files in Forfun as well, that are not sub-directorates. For example, there’s a .txt file lying inside Forfun at the same level as the Test1, Test2 subdirectories.

Every TestX sub-directory is further divided into 3 sub-directories, namely: inputs, results and parameters.

I want to be able to list all the instances of a particular folder like inputs.

The code should recursively go thru the entire Forfun directory and list where all it found the inputs or results directory.

I tried using os.listdir like this:

path = '/myfiles/Forfun'
folders = [f for f in os.listdir(path) if f.endswith('inputs')]
folders 

But this returns me an empty list. Hope I explained the problem clearly, thanks for the help.

Answer 1

os.listdir() lists all files and directories in the provided directory. As you’ve experienced, it does not perform a recursion. It stays on the level of the initially provided directory.

In order to recursively go through a directory tree and iterate through all files and directories below a certain directory level, you should use Python’s os.walk():

import os

def listdirs(suffix):
    for dirname, dirnames, filenames in os.walk('.'):
        for subdirname in dirnames:
            subdirpath = os.path.join(dirname, subdirname)
            if subdirpath.endswith(suffix):
                yield subdirpath


if __name__ == "__main__":
    for d in listdirs('inputs'):
        print d

I’ve set up the same directory structure you’ve been proposing. This is the output of the code above:

$ python test.py 
./Test1/inputs
./Test2/inputs