Question

I would like to learn about randomized trees.
I could not find a good reference about randomized trees.
Can anyone suggest me any good reference or can anyone give me some explanation about randomized trees and how it differs from decision trees?

Answer 1

The idea behind random trees is to use many of them to perform prediction.
This helps the model to be more robust.

The best known algorithm is "Random Forest" proposed by Leo Breiman in 2001 in the following article :
Leo Breiman. Random forests. Machine Learning, 45(1):5–32, October 2001.

Here are the key points of the algorithm:

• each tree of the forest is built on a bootstrap sample of the training data
• at each tree node, the best split is chosen among a reduced set of variable (chosen at random)
• each tree is fully grown (no pruning)
• the prediction is made by a majority vote over all trees