Question

I know what Java Double.NaN is. I have some Java code that produces NaN.

// calculate errors
delta = m1 + m2 - M;
eta = f1 + f2 - F;
for (int i = 0; i < numChildren; i++) {
  epsilon[i] = p[i]*m1+(1-p[i])*m2+q[i]*f1+(1-q[i])*f2-C[i];
}

// use errors in gradient descent
// set aside differences for the p's and q's
float mDiff = m1 - m2;
float fDiff = f1 - f2;
// first update m's and f's
m1 -= rate*delta;
m2 -= rate*delta;
f1 -= rate*eta;
f2 -= rate*eta;
for (int i = 0; i < numChildren; i++) {
  m1 -= rate*epsilon[i]*p[i];
  m2 -= rate*epsilon[i]*(1-p[i]);
  f1 -= rate*epsilon[i]*q[i];
  f2 -= rate*epsilon[i]*(1-q[i]);
}
// now update the p's and q's
for (int i = 0; i < numChildren; i++) {
  p[i] -= rate*epsilon[i]*mDiff;
  q[i] -= rate*epsilon[i]*fDiff;  
}

Under what circumstances will Java produce a NaN value?

Answer 1

Given what I know about gradient descent, you are most probably jumping out to infinity because you do not have adaptive rate (i.e. your rate is too big).