Question

I am playing around with logistic regression in Python. I have implemented a version where the minimization of the cost function is done via gradient descent, and now I’d like to use the BFGS algorithm from scipy (scipy.optimize.fmin_bfgs).

I have a set of data (features in matrix X, with one sample in every row of X, and correpsonding labels in vertical vector y). I am trying to find parameters Theta to minimize:

I have trouble understanding how fmin_bfgs works exactly. As far as I get it, I have to pass a function to be minimized and a set of initial values for Thetas.

I do the following:

initial_values = numpy.zeros((len(X[0]), 1))
myargs = (X, y)
theta = scipy.optimize.fmin_bfgs(computeCost, x0=initial_values, args=myargs)

where computeCost calculates J(Thetas) as illustrated above. But I get some index-related errors, so I think I am not supplying what fmin_bfgs expects.

Can anyone shed some light on this?

Answer 1

After wasting hours on it, solved again by power of posting…I was defining computeCost(X, y, Thetas), but as Thetas is the target parameter for optimization, it should have been the first parameter in the signature. Fixed and works!