I have a relatively simple function with three unknown input parameters for which I only know the upper and lower bounds. I also know what the output Y should be for all of my data.
So far I have done a simple grid search in python, looping through all of the possible parameter combinations and returning those results where the error between Y predicted and Y observed is within a set limit.
I then look at the results to see which set of parameters performs best for each group of samples, look at the trade-off between parameters, see how outliers effect the data etc..
So really my questions is – whilst the grid search method I’m using is a bit cumbersome, what advantages would there be in using Monte Carlo methods such as metropolis hastings instead?
I am currently researching into MCMC methods, but don’t have any practical experience in using them and, in this instance, can’t quite see what might be gained.
I’d greatly appreciate any comments or suggestions
Many Thanks
MCMC methods tend to be useful when the underlying function is complex (sometimes too complicated to directly compute) and/or in high-dimensional spaces. They are often used when nothing else is feasible or works well. Since you have a simple, low-dimensional problem, I wouldn’t expect MCMC approaches to be especially helpful for you.
If you can perform the grid search at a sufficiently-fine scale in a small enough amount of time for your problem domain, it’s likely a good approach.
If your function is convex, there are many well-known approaches such a gradient descent.
If your function has a simple functional form that can easily be solved but you have large amounts of data with gross outliers, RANSAC can be helpful.
If your function has many local minima at unknown locations, simulated annealing can work well.