A change in the distribution of a model's inputs between training and production, so the live features stop looking like the data it learned from. When P(X) moves but P(y | X) stays the same, it is called covariate shift.
Card 1 of 228. Answer: A change in the distribution of a model's inputs between training and production, so the live features stop looking like the data it learned from. When P(X) moves but P(y | X) stays the same, it is called covariate shift.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Data drift is a change in the distribution of the input features, P(X), between the data a model was trained on and the data it scores in production. The learned mapping from inputs to target may still be right, but the model is now asked about regions of feature space it saw rarely or never, where its errors are larger. When P(y | X) stays fixed while P(X) moves, statisticians call it covariate shift.
Example: a credit model is trained on applicants with a median age of 45. A campaign in a student app then brings in a wave of 22-year-olds with short credit histories. How age and history relate to default hasn't changed, but most predictions now fall in a thin corner of the training data, where the model had little to learn from, so overall accuracy drops even though no rule changed.
Detecting it is a two-sample comparison per feature: the Kolmogorov-Smirnov test or Jensen-Shannon distance for continuous features, chi-squared or the population stability index for categorical ones, or a classifier trained to tell old rows from new, where an AUC near 0.5 means no drift. The usual follow-up: drift doesn't always hurt. A shift in a feature the model barely uses can be harmless, so alert on measured or estimated performance, not on drift alone.
