Question

I have a file and suppose I need to split it in up to N smaller files and the smallest chunk should have at least X bytes and all the files should have (almost) the same size:

So using e.g. a string ‘abcdefghij’ with N=4 and X=3 will return [‘abcd’, ‘efg’, ‘hij’] because:

3 chunks < 4 chunks
4 chars > 3 chars

I wrote a split function, but it sometimes create one extra string so I should probably pass the x value instead of calculating there.

def split(string, n):
    x = len(string)//n
    return [string[i:i+x] for i in range(0, len(string), x)]

The real problem is how to calculate the number of chunks to cut the file with a minimum number of bytes.

def calculate(length, max_n, min_x):
    n, x = ...
    return n, x

Is there a simple known algorithm to do this kind of action?

Actually: the files doesn’t need to differ in 1 byte because I want to maximize the number of chunks.

Answer 1

def calculate(L, N, X):
    n = min(L//X, N)
    return n, L//n

Edit:

def spread(seq, N=None, X=1):
    """Yield successive subsequences of seq having at least X elements.

    If N is specified, the number of subsequences yielded will not exceed N.

    The first L % X subsequences yielded (where L = len(seq)) will be longer
    by 1 than the remaining ones.

    >>> list(spread('abcdefghij', 4, 3))
    ['abcd', 'efg', 'hij']
    >>> list(spread('abcdefghijklmnopqrstuvwxyz', 4, 7))
    ['abcdefghi', 'jklmnopqr', 'stuvwxyz']

    seq    any object supporting len(...) and slice-indexing
    N      a positive integer (default: L)
    X      a positive integer not greater than L (default: 1)
    """

    # All error-checking code omitted

    L = len(seq)       # length of seq
    assert 0 < X <= L

    if N is None: N = L
    assert 0 < N

    # A total of n subsequences will be yielded, the first r of which will 
    # have length x + 1, and the remaining ones will have length x.

    # if we insist on using calculate()...
    # n, x = calculate(L, N, X)
    # r = L % n

    # ...but this entails separate computations of L//n and L%n; may as well
    # do both with a single divmod(L, n)
    n = min(L//X, N)
    x, r = divmod(L, n)

    start = 0
    stride = x + 1    # stride will revert to x when i == r
    for i in range(n):
        if i == r: stride = x
        finish = start + stride
        yield seq[start:finish]
        start = finish
    assert start == L