Question

I am looking at the R function gausspr from the kernlab package for Gaussian process regression. The process is defined by the hyperparameters of the kernel function and by the noise in the data. I see in the documentation that I can specify

var: the initial noise variance, (only for regression) (default : 0.001)

but I do not see how to access the estimated value after the regression has run. For instance, consider I have some observed points, and want to predict y values at the locations given by X:

obs <- data.frame(x = c(-4, -3, -1,  0,  2),
                  y = c(-2,  0,  1,  2, -1))
X <- seq(-5,5,len=50)

I can do so with kernlab::gausspr as such:

gp <- gausspr(obs$x, obs$y, kernel="rbfdot", scaled=FALSE, var=.09)
Ef <- predict(gp, X)

I can get the estimated value of the kernel hyperparameter:

gp@kernelf@kpar

But I don’t see how I can return the estimated value of the noise parameter, var?

Answer 1

I might be overlooking something, but I don’t think that the initial noise variance var is “fit” to anything; I don’t think it is a parameter (although I agree that using the word “initial” makes you think otherwise).

The noise variance is just added to the diagonal of the correlation matrix of the training points, as described on this page about some other software. Looking through the function definition, it looks like this is exactly what it is doing in kernlab as well:

# The only relevant line where 'var' is used
alpha(ret) <- solve(K + diag(rep(var, length = m))) %*% y

If you wanted to get the error (or any measure of fit) by the noise variance, you could do something like:

error.fun<-function(x) error(gausspr(obs$x, obs$y, kernel="rbfdot", scaled=FALSE, var=x))
noises<-seq(0.1,1,by=0.1)
y<-sapply(noises,error.fun)
plot(noises,y,type='l')

The built-in cross-validation does not “fit” var in any way, from what I can tell. The only relevant line in the cross validation is here:

cret <- gausspr(x[cind, ], y[cind], type = type(ret), 
                scaled = FALSE, kernel = kernel, var = var, 
                tol = tol, cross = 0, fit = FALSE)

And you can see that var is just put in with no changes.