Removing sub-nodes that do not earn their keep, so a fully grown tree stops fitting noise. Pre-pruning caps growth up front with max depth or minimum samples per leaf; post-pruning grows the tree, then collapses branches whose removal does not hurt held-out error.
Card 1 of 259. Answer: Removing sub-nodes that do not earn their keep, so a fully grown tree stops fitting noise. Pre-pruning caps growth up front with max depth or minimum samples per leaf; post-pruning grows the tree, then collapses branches whose removal does not hurt held-out error.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Pruning removes branches of a decision tree that add complexity without improving predictions on new data, so the tree stops memorizing noise in the training set. It is the main way to control a single tree's overfitting.
Pre-pruning stops the tree from growing in the first place, with limits such as max_depth, min_samples_split, min_samples_leaf or a minimum impurity decrease. Post-pruning grows the full tree and then cuts it back. The standard method is cost-complexity pruning, used by CART and exposed in scikit-learn as ccp_alpha, which trades the tree's error against its number of leaves and picks the strength by cross-validation.
Pruned trees are smaller, faster and easier to explain. Ensembles change the picture: random forests usually grow deep, unpruned trees and rely on averaging to cut variance, while gradient boosting uses shallow trees that are effectively pre-pruned.
