Question

I wrote this piece of code that should make what is described here:

Conjugate Gradient from wikipedia

but after some iterations the variable denomAlpha goes to zero and so I get a NAN on alpha. So what is wrong with my algorithm?

import Jama.Matrix;

public class ConjugateGrad {

    private static final int MAX_IT = 20;
    private static final int MAX_SIZE = 50;

    public static void main(String[] args)  {
        Matrix A = Matrix.random(MAX_SIZE, MAX_SIZE);
        Matrix b = Matrix.random(MAX_SIZE, 1);

        double[][] d = new double[MAX_SIZE][1];
        for(int ii=0;ii<MAX_SIZE;ii++) {
            d[ii][0] =0;
        }

        Matrix x = Matrix.constructWithCopy(d);
        Matrix r = b.minus(A.times(x));
        Matrix p = r;
        Matrix rTrasp_r = r.transpose().times(p);

        for (int i = 0; i < MAX_IT; i++) {
            Matrix denomAlpha = p.transpose().times(A.times(p));
            double numeratorAlpha = rTrasp_r.getArray()[0][0];
            double Alpha = numeratorAlpha / denomAlpha.getArray()[0][0];
            x = x.plus(p.times(Alpha)); 
            r = r.minus(A.times(p));  
            Matrix rNew = r.transpose().times(r); 
            if (Math.sqrt(rNew.getArray()[0][0]) <1.0e-6) {
                break;
            }
            double Beta = rNew.getArray()[0][0] / rTrasp_r.getArray()[0][0];
            p = r.plus(p.times(Beta));
            rTrasp_r = rNew;           
        }
    }
}

it same that with those parameters :

double[][] matrixA = {{4,1},{1,3}};
Matrix A = Matrix.constructWithCopy(matrixA);    
double[][] vectorb = {{1},{2}};
Matrix b = Matrix.constructWithCopy(vectorb);
double[][] d = {{2},{1}};
Matrix x = Matrix.constructWithCopy(d);

at first step of the algorithm things are good
but at second step not…

r: -8.0, -3.0
Alpha: 0.22054380664652568
Beta:   12.67123287671233
x:  0.2356495468277946, 0.33836858006042303, 

Second step :

Alpha: 0.0337280177221555
Beta:  159.11259655226627
x:  -2.2726985108925097, -0.47156587291133856, 

Ok, I have found one Error:

r = r.minus(A.times(p).times(Alpha));  

Now it works:

r: -8.0, -3.0, 
Alpha: 0.22054380664652568
rNew: 0.6403099643121183, 
Beta: 0.008771369374138607
p:  -0.3511377223647101, 0.7229306048685207, 
x:  0.2356495468277946, 0.33836858006042303, 

Answer 1

Sorry for the hack answer but… using the numerical example params from the Wikipedia article and outputting the matrices to terminal at each step could find the discrepancy.