Ensemble Methods

Combining several models so their errors average out, giving lower variance than any single model. Each tree makes different mistakes, so averaging cancels noise while preserving the signal.

When to use

The course example is the Random Forest, which averages many Decision Trees trained on bootstrap samples with random feature subsets. It scored 85% accuracy on the Yalla Fitness churn data versus 78% for a single tree, at the cost of interpretability.

See DS Lec 3.

Appears in