The fundamental difference is, random forest uses bagging technique to make predictions. GBM uses boosting techniques to make predictions.
Card 1 of 259. Answer: The fundamental difference is, random forest uses bagging technique to make predictions. GBM uses boosting techniques to make predictions.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Random forest builds many trees independently and averages them (bagging), while gradient boosting builds trees one after another, each correcting the errors of the ensemble so far (boosting). Both are ensembles of decision trees, but they attack different parts of the error: random forest mainly reduces variance, boosting mainly reduces bias.
In a random forest each tree is grown deep on a bootstrap sample of the rows, and each split considers only a random subset of features, which decorrelates the trees. In gradient boosting each new tree is usually shallow and is fit to the negative gradient of the loss; for squared error that is the residuals. Its output is scaled by a learning rate, such as 0.1.
Random forest trains trees in parallel, needs little tuning, and adding more trees does not cause overfitting. Boosting is sequential and usually more accurate on tabular data when tuned, but sensitive to learning rate, depth and tree count, so it relies on early stopping. XGBoost, LightGBM and CatBoost are popular implementations.
