Decision trees, random forests, and gradient boosting. In order to understand what XGBoost is, we must understand decision trees, random forests, and gradient boosting.