Question

I’d like to create python command line code that is able to print directory tree with sizes of all subdirectories (from certain directory) and most frequent extensions… I will show the example output.

• root_dir (5 GB, jpg (65 %): avi ( 30 %) : pdf (5 %))

— aa (3 GB, jpg (100 %) )

— bb (2 GB, avi (20 %) : pdf (2 %) )

— bbb (1 GB, …)

— bb2 (1 GB, …)

— cc (1 GB, pdf (100 %) )

The format is :

nesting level, directory name (size of the directory with all files and subdirectories, most frequent extensions with size percentages in this directory.

I have this code snippet so far. The problem is that it counts only file sizes in directory, so the resulting size is smaller than real size of the directory. Other problem is how to put it all together to print the tree I defined above without redundant computations.

Answer 1

Calculating directory sizes really isn’t python’s strong suit, as explained in this post: very quickly getting total size of folder. If you have access to du and find, by all means use that. You can easily display the size of each directory with the following line:

find . -type d -exec du -hs "{}" \;

If you insist in doing this in python, you may prefer post-order traversal over os.walk, as suggested by PableG. But using os.walk can be visually cleaner, if efficiency is not the utmost factor for you:

import os, sys
from collections import defaultdict

def walkIt(folder):
    for (path, dirs, files) in os.walk(folder):
        size = getDirSize(path)
        stats = getExtensionStats(files)

        # only get the top 3 extensions
        print '%s (%s, %s)'%(path, size, stats[:3])

def getExtensionStats(files):
    # get all file extensions
    extensions = [f.rsplit(os.extsep, 1)[-1] 
        for f in files if len(f.rsplit(os.extsep, 1)) > 1]

    # count the extensions
    exCounter = defaultdict(int)
    for e in extensions:
        exCounter[e] += 1

    # convert count to percentage
    percentPairs = [(e, 100*ct/len(extensions)) for e, ct in exCounter.items()]

    # sort them
    percentPairs.sort(key=lambda i: i[1])
    return percentPairs

def getDirSize(root):
    size = 0
    for path, dirs, files in os.walk(root):
        for f in files:
            size +=  os.path.getsize( os.path.join( path, f ) )
    return size

if __name__ == '__main__':
    path = sys.argv[1] if len(sys.argv) > 1 else '.'
    walkIt(path)