Study
Machine Learning · Boosting and XGBoost
Boosting builds shallow trees one after another, each fixing the previous errors; gradient boosting, learning rate, early stopping, and XGBoost.
- Bagging averages independent trees; boosting adds trees sequentially, each fitting what is still wrong.
- A small learning rate with early stopping is the standard way to keep boosting from overfitting.