Question

I am trying to generate every combination of a dynamic character set CHAR_LIST, within the range of lower and upper. The code I have pasted below works, but I feel it is horribly inefficient and I would like to make it as fast as possible.

For example, if I want to generate a list between “aab” and “zzz” with only lowercase alphabetic characters it would output: ['aab', 'aac', 'aad', ..., 'zzy', 'zzz']

If there is anything I have left unclear please leave a comment and I will clarify. Thanks!

What I have working right now.

def generate_list(lower, upper):
    result = [lower]
    while lower != upper:
        if CHAR_LIST.index(lower[len(lower)-1:len(lower)]) + 1 < len(CHAR_LIST):
            lower = lower[:len(lower)-1] + CHAR_LIST[CHAR_LIST.index(lower[len(lower)-1:len(lower)]) + 1]
        else:
            new_lower = ""
            new_dig = 0
            inc_next = True
            for i in lower[::-1]:
                if i == CHAR_LIST[len(CHAR_LIST)-1] and inc_next:
                    new_lower += CHAR_LIST[0]
                    new_dig += 1
                else:
                    if inc_next:
                        inc_next = False
                        new_lower += CHAR_LIST[CHAR_LIST.index(i) + 1]
                    else:
                        new_lower += i
            if new_dig == len(lower):
                lower = str(CHAR_LIST[0])*int(len(lower)+1)
            else:
                lower = new_lower[::-1]
        result.append(lower)
    return result

EDIT: I forgot to add, as this is part of the challenge, that it must also compute a list that has different lengths for start and end points. For example it must compute the list between “a” and “zzz” also. Sorry for the late revision, and thanks for the creative answers so far 🙂

Answer 1

It took me quite a while to understand how your code worked, because you’re doing a lot more work than you need to. Here’s an aggressively “pythonized” version of the same algorithm, which I suspect will quite a bit faster than what you have now:

def generate_strings(value, bound, alpha):
    yield value
    while value != bound: # run until we have reached bound
        for i, c in enumerate(reversed(value)): # loop over the string in reverse
            if c != alpha[-1]: # can this character be incremented?
                # construct an incremented value
                value = value[:-1-i] + alpha[alpha.index(c)+1] + alpha[0]*i
                break # exit the for loop
        else: # run only if for loop ended without breaking
            value = alpha[0]*(len(value) + 1) # make a longer string
        yield value

The function is a generator, so if you want a list result, pass it to the list constructor, as in this example output:

>>> print(list(generate_strings("b", "cc", "abcd")))
['b', 'c', 'd', 'aa', 'ab', 'ac', 'ad', 'ba', 'bb', 'bc', 'bd', 'ca', 'cb', 'cc']

I made the sequence of characters an argument to the function, rather than using a global variable. The bound argument can also be None or some other nonsensical value to get an infinite generator (but don’t pass that to list() without shortening it!). Here’s an example of both of those features:

>>> from itertools import islice
>>> from string import ascii_lowercase
>>>
>>> print(list(islice(generate_strings("xyzzy", None, ascii_lowercase), 5)))
['xyzzy', 'xyzzz', 'xzaaa', 'xzaab', 'xzaac']

There are a few things done in the code that might not be obvious if you’re new to Python.

First off, I use a lot of negative indexes into the strings. This counts from the right, starting with -1 as the rightmost character. This alone would simplify your code a lot (you had a lot of x[len(x)-1]).

Next, I use the enumerate and reversed built-in functions to loop over the string from right to left, keeping track of how many characters I’ve looped over. I think this is about what you were doing with your i and new_dig values, but I think it’s much clearer. There are a lot of helpful built in generators in Python!

Finally, I used a break statement to exit the for loop early, with an else block to handle the case where we got to the end without breaking. This sort of else on a loop seemed useless to me when I first learned about it, but it really is handy in situations like this one, where the majority of runs of the loop will result in a break statement being hit.