Question

I’m getting the following error when using fmin_bfgs (in SciPy) to optimize an unregularized logistic cost function:

Divide-by-zero encountered: rhok assumed large

C:\Python27\lib\site-packages\scipy\optimize\optimize.py:828:
RuntimeWarning: divide by zero encountered in double_scalars rhok =
1.0 / (numpy.dot(yk, sk))

Warning: Desired error not necessarily achieved due to precision loss.
Current function value: 0.693147
Iterations: 1
Function evaluations: 27

The algorithm (fmin_bfgs) stops after one iteration. What could I be doing wrong? Here’s the python code: https://gist.github.com/4223554

Here’s the dataset: https://gist.github.com/4223566

Answer 1

folks,

Unlike what ‘pv’ mentioned, the issue wasn’t related to initial_theta. It had to do with the training data set. I’ve fixed the problem and here’s the working code: https://github.com/dormantroot/machine-learning-experiment/blob/master/LogisticRegressionExamples/LogisticRegression.py