When two or more predictors in a regression are strongly correlated, so the model can't separate their individual effects. Their coefficients get large standard errors and can swing or flip sign between samples, though the predictions are often fine.
Card 1 of 244. Answer: When two or more predictors in a regression are strongly correlated, so the model can't separate their individual effects. Their coefficients get large standard errors and can swing or flip sign between samples, though the predictions are often fine.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
With correlated predictors, many combinations of coefficients fit the data almost equally well, so the estimates are unstable. Mathematically, XᵀX is close to singular, which inflates the variance of the coefficients. In the extreme case, perfect multicollinearity, one column is an exact linear combination of others and the coefficients are not unique at all.
Example: the variance inflation factor measures it. Regress each predictor on all the others and compute VIF = 1/(1 − R²). If a house's floor area and its number of rooms predict each other with R² = 0.9, each gets a VIF of 10: its coefficient's variance is 10 times what it would be with uncorrelated predictors. Values above 5 to 10 are the usual warning sign.
Whether to act depends on the goal. For prediction, multicollinearity matters little as long as the correlation between predictors holds in new data. For interpreting coefficients or feature importance it matters a lot. Fixes: drop or combine correlated features, use principal components, use ridge regression, which stabilizes the coefficients, or collect more varied data. The dummy variable trap, one-hot encoding every category alongside an intercept, is the classic perfect case.
