Lets say we have an array of a million elements, and we want to accept user input on what kind of math to apply to each of those elements.
Would the program need to evaluate the user formula (string) a million times, for each of the elements. Or can the formula itself somehow be saved and interpreted only once? And then be applied in a loop to the million elements?
I’m just trying to get a general idea on how this works. Because right now it looks like excel type programs are always interpreted which is way too slow for my data research. So basically what I’m doing is re-compiling my user defined math each and every time I change it. And there seems to be no way of compiling a program, and recompiling it while it itself is running. So that it could accept user input, compile it, and run it, without quitting. Maybe with DLLs and seperate app domains, but then all data has to be marshalled.
I just don’t get how one gets top speed for data research, when everything is going against you. Including the stupid operating system that can’t allow the DLL to be unloaded, loaded, because of security concerns. Or something. However maybe I’m just spurting nonsense, since I just began programming a few months ago.
In case you are interested in, or prefer to implement the solution yourself, you could parse the formula into a (binary) tree, where each node represents an operator (+, -, *,..) or a function (cos(), max(),..) and each leaf represents either a constant or one of the formula’s variables.
This can be done in a pretty efficient way and then applied to the entire data set by recursively calculating the value of the root while filling in the correct variables at the leaves.
To figure out how best to do this, this question might provide some pointers (though I am fairly certain the answer provided there is only one of several ways):
Create Binary Tree from Mathematical Expression
Hope this helps or at least is of some interest to other readers!