Question

An application that writes data to disk in 100MB chunks and increments the file-name by appending +1, so n1, n2 ... n1000. This eventually uses all of the free space on the partition (Linux host). I am looking for a way to delete files that were first written in the series until the drive space is under a specific utilization.

If the latter, would https://stackoverflow.com/a/5912404/666891 be a good solution?

The following solution was proposed and looks to be a viable solution per https://stackoverflow.com/a/837840/666891. How could this be modified to handle the incrementing file extension as currently when the script is run it does not delete files name filename*, asterisk being the incrementing number, start with the oldest one?

import os
def free_space_up_to(free_bytes_required="161061273600", rootfolder="/data/", ex
tension="filename-*"):
    file_list= files_to_delete(rootfolder, extension)
    while file_list:
        statv= os.statvfs(rootfolder)
        if statv.f_bfree*statv.f_bsize >= free_bytes_required:
            break
        os.remove(file_list.pop())

Answer 1

Well, if you know that all files are (at least sort of) 100MB in size, and assuming there’s nothing else drastically altering disk usage on the machine, you don’t need to check for free space at every iteration.

Also, if all files have the same name, besides the counter at the end, you can skip the os.stat call (which could also be useless for files created in quick succession) and sort on the filenames based on the counter:

import os

def free_space_up_to(free_bytes_required=161061273600, rootfolder="/data/", filesize=104857600, basename="filename-"):
    '''Deletes rootfolder/basename*, oldest first, until there are free_bytes_required available on the partition.
    Assumes that all files have file_size, and are all named basename{0,1,2,3,...}
    Returns number of deleted files.
    '''
    statv = os.statvfs(rootfolder)
    required_space = free_bytes_required - statv.f_bfree*statv.f_bsize
    basepath = os.path.join(rootfolder, basename)
    baselen = len(basepath)
    if required_space <= 0:
        return 0

    # "1 +" here for quickly rounding
    files_to_delete = 1 + required_space/filesize

    # List all matching files. If needed, replace with os.walk for recursively
    # searching into subdirectories of rootfolder
    file_list = [os.path.join(rootfolder, f) for f in os.listdir(rootfolder)
                 if f.startswith(basename)]

    file_list.sort(key=lambda i: int(i[baselen:]), reverse=True)
    # Alternatively, if the filenames can't be trusted, sort based on modification time
    #file_list.sort(key=lambda i: os.stat(i).st_mtime)

    for f in file_list[:files_to_delete]:
        os.remove(f)
    return files_to_delete

(Not thoroughly tested, I recommend a test run substituting “os.remove” with “print” ;))