Question

Classification and regression trees by Breiman et al mentioned about using linear combination of predictors when splitting a node. I was trying to find a way to try this out with R in vain.

There are tree or rpart packages which assume splitting on an univariate predictor and they do not allow any customization with linear combination. Do I have to create my own package?

Answer 1

I haven’t used it before, but you might have a look at the oblique.tree package on CRAN.

The example in ?oblique.tree does in fact add the PCA output at covariates,

data(crabs, package = "MASS")
aug.crabs.data <- data.frame(   g=factor(rep(1:4,each=50)),
                predict(princomp(crabs[,4:8]))[,2:3])

yielding data that looks like this:

where the covariates are the 2nd and 3rd principal components. It can then apparently fit a decision tree that splits on linear combinations of these variables:

ob.tree <- oblique.tree(formula     = g~.,
            data        = aug.crabs.data,
            oblique.splits  = "only")
plot(ob.tree)
text(ob.tree,cex = 0.5)