Question

I have a directory with a few thousand files. I’d like to be able to make sub directories inside the larger director and move exactly 150 files (filtered by extension) into each of the newly created directories. I would love some guidance to get me going. Here is the trifle I’m starting with:

wd = os.chdir(r'foobar')

allgzs=[]

for file in os.listdir(wd):
    if file.endswith('asc.gz'):
       allgzs.append(file)

for gz in allgzs:


# For every 150 .gz files: 
# Create directory -- os.mkdir?
# Move files into directory  -- shutil.move?

Answer 1

I came up with a grouper function to yield a long sequence in runs of n:

import itertools
def grouper(S, n):
    iterator = iter(S)
    while True:
        items = list(itertools.islice(iterator, n))
        if len(items) == 0:
            break
        yield items

From there, you can do what you want pretty concisely:

import glob, os, shutil
fnames = sorted(glob.glob('*asc.gz'))
for i, fnames in enumerate(grouper(fnames, 150)):
    dirname = 'batch%d' % i
    os.mkdir(dirname)
    for fname in fnames:
        shutil.move(fname, dirname)