Question

I have a function which is merging certain files from a directory

def merge(path):
    f = open("indexFile","w")
    for path,directory,files in os.walk(path):
        for file in files:
            f1 = open(os.path.join(path,file))
            createCatFile(f1.read())
            print "merging files"
            shutil.copyfileobj(f1, f)
            f1.close()
    f.close()

Before copying the file object, it is passing the contents of f1 to a function to do some processing. The problem is that the indexFile is created but there is not data in the file. it is an empty file. The createCatFile() function works perfectly as expected. Also, “merging files” is printed number of times the merge() function is called. When I remove the function call to createCatFile() the indexFile is created successfully.

Any help as to what is the problem with this?

The createCatFile function does the following:

def createCatFile(wordtodocstr):
    global offset
    wordInfo = wordtodocstr.split()
    term = wordInfo[0]
    newtermid = wordInfo[1]
    docList = wordInfo[2::2]
    ctfList = [int(number) for number in wordInfo[3::2]]
    docfr = len(docList)
    wordctf = sum(ctfList)
    catFileList = [term, newtermid, str(offset), str(wordctf), str(docfr)]
    catFileJoin = " ".join(catFileList)
    with open(path2+term, "w") as foutterm:
        foutterm.write(catFileJoin)
        foutterm.close()    
    offset+=1

Thank you.

Answer 1

Maybe doing f1.read() “consumes” the file, and nothing is left for shutil to copy. Try f1.seek(os.SEEK_SET) to rewind the file before calling copyfileobj