A family of ensemble methods that trains weak learners in sequence, each one focusing on the examples the previous ones got wrong, and combines them in a weighted vote or sum to form a strong learner. AdaBoost and gradient boosting are the best-known examples.
Card 1 of 259. Answer: A family of ensemble methods that trains weak learners in sequence, each one focusing on the examples the previous ones got wrong, and combines them in a weighted vote or sum to form a strong learner. AdaBoost and gradient boosting are the best-known examples.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Boosting is a family of ensemble methods that trains weak learners in sequence, each focusing on the examples the ensemble so far gets wrong, then combines them in a weighted vote to form a strong learner. A weak learner, such as a one-split stump, only needs to beat random guessing.
In binary AdaBoost each example carries a weight. A stump with weighted error ε = 0.3 gets a vote of α = ½ ln((1 − ε)/ε) ≈ 0.42; the weights of the examples it got wrong are multiplied by e^α ≈ 1.53, the rest by e^−α ≈ 0.65, then all are renormalized. The mistakes go from 30% of the weight to exactly 50%, so the next stump targets them.
It came from theory: Kearns and Valiant asked in the late 1980s whether weak learners can always be made strong, Schapire proved it in 1990, and Freund and Schapire introduced AdaBoost in 1995. Boosting mainly cuts bias but can overfit noisy labels; tune the number of rounds, often by early stopping.
