Question

With the below code I receive IOError: [Errno 13] Permission denied, and I know this is due to the output directory being a sub-folder of the input directory:

import datetime
import os

inputdir = "C:\\temp2\\CSV\\"
outputdir = "C:\\temp2\\CSV\\output\\"
keyword = "KEYWORD"

for path, dirs, files in os.walk(os.path.abspath(inputdir)):
    for f in os.listdir(inputdir):
        file_path = os.path.join(inputdir, f)
        out_file = os.path.join(outputdir, f)
        with open(file_path, "r") as fh, open(out_file, "w") as fo:
            for line in fh:
                if keyword not in line:
                    fo.write(line)

However, when I change the output folder to: outputdir = "C:\\temp2\\output\\" the code runs successfully. I want to be able to write the modified files to a sub-folder of the input directory. How would I do this without getting the ‘Permission denied’ error? Would the tempfile module be useful in this scenario?

Answer 1

os.listdir will return directory as well as file names. output is within inputdir so the with is trying to open a directory for reading/writing.

What exactly are you trying to do? path, dirs, files aren’t even being used in the recursive os.walk.

Edit: I think you’re looking for something like this:

import os

INPUTDIR= "c:\\temp2\\CSV"
OUTPUTDIR = "c:\\temp2\\CSV\\output"
keyword = "KEYWORD"

def make_path(p):
    '''Makes sure directory components of p exist.'''
    try:
        os.makedirs(p)
    except OSError:
        pass

def dest_path(p):
    '''Determines relative path of p to INPUTDIR,
       and generates a matching path on OUTPUTDIR.
    '''
    path = os.path.relpath(p,INPUTDIR)
    return os.path.join(OUTPUTDIR,path)

make_path(OUTPUTDIR)

for path, dirs, files in os.walk(INPUTDIR):
    for d in dirs:
        dir_path = os.path.join(path,d)
        # Handle case of OUTPUTDIR inside INPUTDIR
        if dir_path == OUTPUTDIR:
            dirs.remove(d)
            continue
        make_path(dest_path(dir_path))    
    for f in files:
        file_path = os.path.join(path, f)
        out_path = dest_path(file_path)
        with open(file_path, "r") as fh, open(out_path, "w") as fo:
            for line in fh:
                if keyword not in line:
                    fo.write(line)