An ensemble of many decision trees, each trained on a bootstrap sample of the rows and choosing each split from a random subset of features. It predicts by majority vote or by averaging the trees' outputs; it doesn't pick one best tree.
Card 1 of 276. Answer: An ensemble of many decision trees, each trained on a bootstrap sample of the rows and choosing each split from a random subset of features. It predicts by majority vote or by averaging the trees' outputs; it doesn't pick one best tree.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
A random forest is an ensemble of many decision trees whose predictions are combined: a majority vote for classification, an average for regression. It does not pick one best-fitting tree or build an "average" tree, a common misreading; every tree is kept. One deep tree overfits; many different ones averaged generalize far better.
The trees differ because each is grown on a bootstrap sample of the rows, and each split may only consider a random subset of the features, often √p of p, so 8 of 64. Example: 500 trees predict whether a customer will churn: 340 vote yes and 160 no, so the forest predicts churn with a score of 0.68.
Common follow-ups: why it needs little tuning (more trees cost time but do not overfit), and why impurity-based feature importance favors features with many distinct values, making permutation importance safer. A surprising limit: a regression forest outputs averages of training targets, so it can never predict outside the range of targets it was trained on.
