A mismatch between how features are computed for training and how they are computed when the model serves predictions, so the live model sees inputs that differ from what it learned from. It is a pipeline bug, not a change in the world.
Card 1 of 228. Answer: A mismatch between how features are computed for training and how they are computed when the model serves predictions, so the live model sees inputs that differ from what it learned from. It is a pipeline bug, not a change in the world.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Training-serving skew is a difference between a model's performance or inputs at training time and at serving time caused by the system itself. Common sources: feature logic written twice (a SQL batch job for training, service code for serving), different data sources or time windows, preprocessing applied in one path but not the other, and training data that includes information the live system doesn't have yet.
Example: a model uses "orders in the last 30 days". The training job counts orders in the warehouse, where refunded orders have already been removed. The online service counts orders from the event stream, where refunds arrive days later. Live values run higher than anything comparable in training, and the model systematically overrates active customers. Offline validation looks fine, because it was computed with the training logic.
Prevention: define each feature once and use it in both paths (a shared transformation library or a feature store), log the features actually served and train on those logs, and compare training and serving values for the same entities. Rule #29 of Martin Zinkevich's Rules of Machine Learning at Google says it directly: save the features used at serving time and use them for training.
