Question

A script I wrote was meant to move individual day directories from an old location to a new location with a structure like this:

/old/YYYY/MM/DD
/new/YYYY/MM/DD

and for another task (unrelated to the moving of data) I created a softlink in the new location like this (this was the first mistake I made):

/new/2011/09 -> /old/2011/09

My script essential used this function call:

for d in os.listdir("/old/2011/09"):
    shutil.move(os.path.join("/old/2011/09/", d), os.path.join("/new/2011/09", d))

After running my script 2011/09 was empty in both. I had this occur at work with unarchived data…big problem. My question is why didn’t shutil.move() give me an error that the day directory I was moving already existed? Each day inside 09 should have been the same directory because of the softlink.

/new/2011/09/01 == /old/2011/09/01

Doesn’t the shutil.move call check the src and dst before calling shutil.copy2? From the docs: “The destination directory must not already exist.” or is that only when it uses rename? AND if it makes a difference both old and new locations were glusterfs.

shutil.move Documentation

shutil.copy Documentation

Thanks for any clarity you can provide.

EDIT/UPDATE: I submitted a question to the python-list asking why this behavior existed and asked if it should be changed (list archive). They suggested I file a bug report. While doing the tests to submit the bug I found out that this has been fixed in Python 2.7. You can see the differences in the source in the move function declaration: Python 2.6 and Python 2.7.

This still does the move/rename but won’t magically delete an entire directory.

Answer 1

It comes down to these two lines in shutil.move:

        copytree(src, real_dst, symlinks=True)
        rmtree(src)

where src='old' and real_dst='new/old'. The copytree command copies old to the subdirectory new/old. That goes fine, although it may not be what you intended.

rmtree removes the old directory. That’s a problem, since new is now a dangling symlink.