When information from outside the training dataset is used to create the model. For example, including any information from the validation or test sets into the model.
Card 1 of 276. Answer: When information from outside the training dataset is used to create the model. For example, including any information from the validation or test sets into the model.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Data leakage is when information that would not be available at prediction time gets into the training data or the features. The model looks great in validation and then fails in production, because it learned from a signal it will never see again.
The two common forms are target leakage and train-test contamination. Target leakage is a feature that is a consequence of the label, such as "days in hospital" when predicting whether a patient will be admitted, or a refund flag when predicting fraud. Contamination happens when preprocessing such as scaling, imputation or feature selection is fit on the full dataset before splitting, or when random splits put the same customer, or future rows of a time series, on both sides.
To prevent it: split before any fitting, put all preprocessing inside a pipeline that is fit on training folds only, split by time or by group when rows are related, and be suspicious of any single feature with implausibly high importance or validation scores that seem too good to be true.
