Not standard shuffled k-fold: time series rows are ordered and autocorrelated, so random folds train on the future to predict the past. Use time-ordered splits instead, such as walk-forward validation (scikit-learn's TimeSeriesSplit).
Card 1 of 259. Answer: Not standard shuffled k-fold: time series rows are ordered and autocorrelated, so random folds train on the future to predict the past. Use time-ordered splits instead, such as walk-forward validation (scikit-learn's TimeSeriesSplit).
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Not in its standard form. Ordinary k-fold cross-validation shuffles rows into k folds and assumes they are independent and interchangeable. Time series rows are ordered and autocorrelated, so shuffled folds train on the future to predict the past, with each test point's neighbors in the training set, and scores come out optimistic.
Use forward-chaining, or walk-forward, validation instead: every training window ends before its test window begins. With 60 months of data, scikit-learn's TimeSeriesSplit with five splits trains on months 1 to 10, tests on 11 to 20, then trains on 1 to 20, tests on 21 to 30, and so on up to training on 1 to 50 and testing on 51 to 60.
Follow-ups: leave a gap between train and test when features use lags or labels look ahead, match the test window to the forecast horizon, and use a fixed-length rolling window if old data goes stale. A nuance: Bergmeir, Hyndman and Koo (2018) showed standard k-fold is valid for purely autoregressive models with uncorrelated errors.
