Question

Apologies if this is better suited in CrossValidated.

I am fitting GAM models to binomial data using the mgcv package in R. One of the covariates is periodic, so I am specifying the bs = "cc" cyclic cubic spline. I am doing this in a cross validation framework, but when I go to fit my holdout data using the predict function I get the following error:

Error in pred.mat(x, object$xp, object$BD) : 
  can't predict outside range of knots with periodic smoother

Here is some code that should replicate the error:

# generate data:
x <- runif(100,min=-pi,max=pi)
linPred <- 2*cos(x) # value of the linear predictor
theta <- 1 / (1 + exp(-linPred)) # 
y <- rbinom(100,1,theta)
plot(x,theta)
df <- data.frame(x=x,y=y)

# fit gam with periodic smoother:
gamFit <- gam(y ~ s(x,bs="cc",k=5),data=df,family=binomial())
summary(gamFit)

plot(gamFit)

# predict y values for new data:
x.2 <- runif(100,min=-pi,max=pi)
df.2 <- data.frame(x=x.2)
predict(gamFit,newdata=df.2)

Any suggestions on where I’m going wrong would be greatly appreciated. Maybe manually specifying knots to fall on -pi and pi?

Answer 1

I did not get an error on the first run but I did replicate the error on the second try. Perhaps you need to use set.seed(123) #{no error} and set.seed(223) #{produces error}. to see if that creates partial success. I think you are just seeing some variation with a relatively small number of points in your derivation and validation datasets. 100 points for GAM fit is not particularly “generous”.

Looking at the gamFit object it appears that the range of the knots is encoded in gamFit$smooth[[1]]['xp'], so this should restrict your inputs to the proper range:

 x.2 <- runif(100,min=-pi,max=pi); 
 x.2 <- x.2[findInterval(x.2, range(gamFit$smooth[[1]]['xp']) )== 1]

 # Removes the errors in all the situations I tested
 # There were three points outside the range in the set.seed(223) case