Decision Trees

---

Ensemble Methods

Bagging proceeds by replicating the original training set many times. It then fits a separate decision tree to each copy, and combines all trees to create a single predictive model.

Boosting operates similarly, but instead of fitting each model in parallel, it does so in sequence. Therefore, each successive iteration of the model will depend upon those prior.

• Boosting learns slowly, rather than hard-fitting the data with a single, large decision tree.

• Furthermore, instead of fitting the model to the response variable, Y, boosting fits to the residuals of the prior model.