I have a table with a large amount of data and need to do lookups on each and break down each of the data. Here’s a simplified numeric example. I have this table:
1 [1]
2 [1, 1]
4 [2, 2]
now I want to break down 4. I look up and see 2+2=4. so then I look up 2 and see if breaks down into 1+1 so I know 2+1+1=4 and 1+1+1+1=4. For this problem, I should break it down(using the computed table) into 4 results(the 3 mentioned and 4 *1 =4).
I am not sure but is this a graph problem? or some other type? I think I can solve this by just using a recursions that break this down, but I want to learn if there’s an general accepted way and this process will deal with large amounts of data so I’ll need to design it in a way that the breakdown can be distributed over multiple CPUs.
Any idea what type of problem this is or the logic to solve it?
As close as I can understand your specific example, it could be recursion, it could be a graph problem, it could be several other things, or a combination. Not every programming problem can be sorted into a single neat category, and there are generally at least a half-dozen different valid approaches to any problem.
In terms of dealing with large amounts of data specifically, there are many, many different strategies that may be employed, depending on how it needs to be accessed (sequentially? randomly by offset? randomly by key or some sort of search?), how frequently it will be updated, how much data there is in relationship to the sizes of the various levels of the storage hierarchy, etc.
And then there’s multiple CPUs — parallel processing — where data synchronization becomes an important issue, in addition to the other problems.