Question

I have a fairly complex function f(x) that I want to optimize and I am using the fmin_bfgs function from the scipy.optimize module from Scipy. It forces me to give the function to minimize and the function of the gradient f'(x) separately, which is a pity because some of the computations for the gradient can be done when evaluating the function f(x).

Is there a way of combining both functions? I was considering saving the intermediate values required for both functions, but I don’t know if the fmin_bfgs function guarantees that f(x) is evaluated before than f'(x).

Thank you

Answer 1

You could use memoization to cache the intermediate values. Regardless of which function is called first, the second will be able to take advantage of the memoized value.

---

cache={}
def expensive_calc(x):
    # If x is a numpy array, you need to convert x into something hashable so it
    # can be used as a key in cache.
    key=tuple(x)
    try:
        return cache[key]
    except KeyError:
        # do expensive calc for result
        cache[key] = result
        return result

def func(x):
    y=expensive_calc(x)
    return something_based_on_y

def func_der(x):
    y=expensive_calc(x)
    return something_else_based_on_y

def optimize_fmin_bfgs():
    xopt = fmin_bfgs(func, x_guess, fprime=func_der)