Question

UPDATE: Based on rrenaud’s great comment I wanted to clarify that although I’m generating all permutations the order does not matter(so its not exactly like a traveling salesman problem..but still a bit similar). The observation I was focusing on my question(although any insights on how to speed this up are welcomed), was that the results are building upon each other and perhaps there are methods(dynamic programming?) that I could use to build upon previous results as opposed to repeat a lot of calculations. Look at these results and you can see the results are just building on each other(to do this to code change number of items to 3 and in the for loop max to 3):

Grapes = 0
Strawberries = 0
Raspberries = 0
****
Grapes = 1
Strawberries = 0
Raspberries = 0
****
Grapes = 2
Strawberries = 0
Raspberries = 0
****
Grapes = 3
Strawberries = 0
Raspberries = 0
****
Grapes = 0
Strawberries = 1
Raspberries = 0
****
Grapes = 1
Strawberries = 1
Raspberries = 0
****
Grapes = 2
Strawberries = 1
Raspberries = 0
****
Grapes = 3
Strawberries = 1
Raspberries = 0
****
Grapes = 0
Strawberries = 2
Raspberries = 0
****
Grapes = 1
Strawberries = 2
Raspberries = 0
****
Grapes = 2
Strawberries = 2
Raspberries = 0
****
Grapes = 3
Strawberries = 2
Raspberries = 0
****
Grapes = 0
Strawberries = 3
Raspberries = 0
****
Grapes = 1
Strawberries = 3
Raspberries = 0
****
Grapes = 2
Strawberries = 3
Raspberries = 0
****
Grapes = 3
Strawberries = 3
Raspberries = 0
****
Grapes = 0
Strawberries = 0
Raspberries = 1
****
Grapes = 1
Strawberries = 0
Raspberries = 1
****
Grapes = 2
Strawberries = 0
Raspberries = 1
****
Grapes = 3
Strawberries = 0
Raspberries = 1
****
Grapes = 0
Strawberries = 1
Raspberries = 1
****
Grapes = 1
Strawberries = 1
Raspberries = 1
****
Grapes = 2
Strawberries = 1
Raspberries = 1
****
Grapes = 3
Strawberries = 1
Raspberries = 1
****
Grapes = 0
Strawberries = 2
Raspberries = 1
****
Grapes = 1
Strawberries = 2
Raspberries = 1
****
Grapes = 2
Strawberries = 2
Raspberries = 1
****
Grapes = 3
Strawberries = 2
Raspberries = 1
****
Grapes = 0
Strawberries = 3
Raspberries = 1
****
Grapes = 1
Strawberries = 3
Raspberries = 1
****
Grapes = 2
Strawberries = 3
Raspberries = 1
****
Grapes = 3
Strawberries = 3
Raspberries = 1
****
Grapes = 0
Strawberries = 0
Raspberries = 2
****
Grapes = 1
Strawberries = 0
Raspberries = 2
****
Grapes = 2
Strawberries = 0
Raspberries = 2
****
Grapes = 3
Strawberries = 0
Raspberries = 2
****
Grapes = 0
Strawberries = 1
Raspberries = 2
****
Grapes = 1
Strawberries = 1
Raspberries = 2
****
Grapes = 2
Strawberries = 1
Raspberries = 2
****
Grapes = 3
Strawberries = 1
Raspberries = 2
****
Grapes = 0
Strawberries = 2
Raspberries = 2
****
Grapes = 1
Strawberries = 2
Raspberries = 2
****
Grapes = 2
Strawberries = 2
Raspberries = 2
****
Grapes = 3
Strawberries = 2
Raspberries = 2
****
Grapes = 0
Strawberries = 3
Raspberries = 2
****
Grapes = 1
Strawberries = 3
Raspberries = 2
****
Grapes = 2
Strawberries = 3
Raspberries = 2
****
Grapes = 3
Strawberries = 3
Raspberries = 2
****
Grapes = 0
Strawberries = 0
Raspberries = 3
****
Grapes = 1
Strawberries = 0
Raspberries = 3
****
Grapes = 2
Strawberries = 0
Raspberries = 3
****
Grapes = 3
Strawberries = 0
Raspberries = 3
****
Grapes = 0
Strawberries = 1
Raspberries = 3
****
Grapes = 1
Strawberries = 1
Raspberries = 3
****
Grapes = 2
Strawberries = 1
Raspberries = 3
****
Grapes = 3
Strawberries = 1
Raspberries = 3
****
Grapes = 0
Strawberries = 2
Raspberries = 3
****
Grapes = 1
Strawberries = 2
Raspberries = 3
****
Grapes = 2
Strawberries = 2
Raspberries = 3
****
Grapes = 3
Strawberries = 2
Raspberries = 3
****
Grapes = 0
Strawberries = 3
Raspberries = 3
****
Grapes = 1
Strawberries = 3
Raspberries = 3
****
Grapes = 2
Strawberries = 3
Raspberries = 3
****
Grapes = 3
Strawberries = 3
Raspberries = 3
****

I’m still learning algorithms so my arsenal of knowledge is a bit limited. I basically have a recursion that keeps growing exponentially as I add more inputs to it but I’m wondering if there’s anything I can use to make it not do that.

Here’s a sample(java code), of a recursion that basically takes a list of items and figures out all the combinations of each item using 0-n quantities(for simplicity my code has 2 items and each item has quantities of 0 to 5..variables can be changed in the loop). To make this more interesting I have a variable called Q that does some random processing and checks its value against a list and only then proceeds. I commented the code as well as I could so hopefully it reads easily for you:

import java.util.Arrays;
import java.util.List;
//learning playground safe to delete

public class main {

    public static void main(String[] args) {
        System.out.println("Starting..");
        Integer number_of_items = 2; //how many items should we test with, over 7 or 8 takes a long time MAX is 11(based on the number of names we have below)
        long startTime = System.currentTimeMillis(); //start timer
        Integer[] integers_temp = new Integer[number_of_items]; // create a list with exactly the number of items specified above
        Arrays.fill(integers_temp, 0); // populate list with zeros 
        List<Integer> list_to_start = Arrays.asList(integers_temp); //set it as a list
        String[] name_of_list_to_start = new String[] {"Grapes", "Strawberries", "Raspberries", "Blackberries", "Pineapples", "Oranges", "Prunes", "Pears", "cherries", "Peaches", "Apples"};       
        List<Integer> numbers_to_choose_from = Arrays.asList(new Integer[] {0, 1,2,3,4,5,6,7,8,9,10}); //list of numbers program can choose from(could be anything,just learning)
        counter(list_to_start.size(), list_to_start, name_of_list_to_start, numbers_to_choose_from);
        long endTime = System.currentTimeMillis(); 
        System.out.println("Total execution time: " + (endTime-startTime));
    }

    private static void counter(int length, List<Integer> list_to_start, String[] name_of_list_to_start, List<Integer> numbers_to_choose_from) {
        // If we've gone through everything then return the results
        if (length == 0) {
            for (int i = 0; i<list_to_start.size(); i++) {
                System.out.println(name_of_list_to_start[i] + " = " + list_to_start.get(i));
            }
            System.out.println("****");
            return;
        }   
        //This part basically increments list_to_start and then the above part displays it.
        for (int i = 0; i<=5; i++) {
            int q = i +2;  //do anything here..random just for example, right now just takes the number in the loop and adds by 10
            //System.out.println(q);  // this area is looped as many times as i^items, yet seems like after the first run work is duplicated.
            if (length != 0 && numbers_to_choose_from.contains(q)) {
                list_to_start.set((length-1), q);
                counter((length-1), list_to_start, name_of_list_to_start, numbers_to_choose_from);
                list_to_start.set((length-1), 0);
            }
        }
    }
}

If you change the quantities to 0-10 and the items to 10 then it becomes (10^10 which is 10000000000 loops). When I profile the code it says most of the time is spent on the iterator, which makes sense so I need to somehow reduce the time it spends iterating(hopefully there’s a way to get it to iterate once per item instead of on every recursion).

My observation is the initial for loop seems necessary because its calculating and checking the values against the list but once that is done all the other times it loops is repeating the same process. I’m very unfamiliar all the algorithms (but currently reading intro to algorithms), so I’m wondering if there’s a way I could make this no exponential so it can handle more variables(this is a part of my program which is much larger, but ideally I would need to process as many as possible but at least 0-100 quantities of 100 items and have it done within an hour or other jobs fail. There’s no way I can achieve this with 100^100, so I’m struggling for ideas).

If anyone has any tips or if they have seen any algorithms that solve a similar issue, that would be great because I could study the logic they use and see if it applies. If anyone has a direct solution then thats great too!

I hope this question is not too long and makes sense(I tried to be as detailed as possible)

Answer 1

If you want generate all possible permutations, this belongs to EXPTIME-complete class like towers of Hanoi, its more difficult than NP-complete problems (because all of them belong to PSPACE), you can’t expect faster than 100^100. But may be there is some heuristics for original problem, So it’s better to say it. Also if your original problem is this, you can say your boss: No one in the world can do this. Also you can make it parallel, but except using extremely powerful mainframe, you don’t have any chance to get result.