Bagging trains models independently on bootstrap samples and averages or votes their predictions, which mainly reduces variance. Boosting trains weak learners one after another, each focusing on the errors of the ensemble so far, and combines them in a weighted sum, which mainly reduces bias.
Card 1 of 259. Answer: Bagging trains models independently on bootstrap samples and averages or votes their predictions, which mainly reduces variance. Boosting trains weak learners one after another, each focusing on the errors of the ensemble so far, and combines them in a weighted sum, which mainly reduces bias.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Bagging trains many models independently, each on its own bootstrap sample, and combines them with an equal-weight average or vote. Boosting trains models one after another, each focusing on the examples the ensemble still gets wrong, and combines them in a weighted sum. Bagging mainly reduces variance; boosting mainly reduces bias.
AdaBoost shows the sequential part. Give 10 training points weight 0.1 each. The first stump misclassifies 3, a weighted error ε = 0.3, so its vote is α = ½ ln((1 − ε)/ε) ≈ 0.42. Misclassified weights are multiplied by exp(0.42) ≈ 1.53, the rest by exp(−0.42) ≈ 0.65; after renormalizing, the 3 hard points hold 50% of the weight, up from 30%.
This decides the base learner. Bagging needs strong, unstable models such as deep trees, since averaging cannot remove errors every model shares; weak learners such as stumps belong to boosting. Bagging runs in parallel and adding models does not overfit; boosting is sequential, needs a learning rate and early stopping, and can chase mislabeled points.
