There’s a little script that grabs a bunch of data from a database, and does an iterative calculation. There are about 2500 rows used in this calculation, so it’s not a huge amount, but my boss wants me to partition the calculation anyways (as an exercise).
My general strategy (and I’m just shooting in the dark) is to hit the database, grab the first 50 rows, do each step in the calculation for those 50 rows, store the last row (as the calculation is iterative), grab the next 50 rows from the database and continue this process until all of the rows in the database have been accounted for.
Thoughts on my strategy? Any tips for doing this sort of thing?
One of the first things I learned in programming is that when you don’t know how to code something, first write out the process (algorithm) you’d use to solve it yourself, step-by-step, then see how to translate that to code.
Sounds like a good first step for you would be to write out how you would solve the problem on paper–without worrying about partitioning. I know your problem isn’t this trivial, but I’m going to use an example of summation.
To find the total of all the records, you would take record0 + record1 + record2 + … + record2499 = Sum.
With that down, you can then go about seeing if it can be partitioned. For addition, that is easily done because addition is associative. Group up operations, and that’s one partition.
Now, if you can’t find a way to partition the calculation manually, then it’s going to be difficult to try to partition it in code.
But, my first step would be to work it out manually, then look for partition possibilities there.