Overfitting: the model has memorised noise particular to the training set, so training error keeps falling while validation error turns back up. The fixes matter as much as the name: more data, regularization, early stopping, a simpler model, and cross-validation to detect it.
Card 1 of 259. Answer: Overfitting: the model has memorised noise particular to the training set, so training error keeps falling while validation error turns back up. The fixes matter as much as the name: more data, regularization, early stopping, a simpler model, and cross-validation to detect it.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Overfitting. A model overfits when it learns the noise in its training set along with the real signal, so it scores well on the training data and badly on new data. The telltale sign is a gap: training error keeps falling while validation error stalls and then rises.
Example: fit a degree-9 polynomial to 10 noisy points from a straight line. The curve passes through every point, so training error is zero, but it swings wildly between points and predicts new values poorly; a straight line has some training error but generalizes far better. The overfit model has low bias and high variance: small changes in the training data change it a lot.
Detecting and fixing it is the usual follow-up. Detect it with a held-out validation set, cross-validation or learning curves. Fix it with more data, regularization such as L1, L2 or dropout, early stopping or a simpler model. Don't confuse it with data leakage: a leaky model also looks good on validation, so the gap never appears.
