Overlapping Predictors in Regression: Stability Diagnostics Beyond VIF

What is multicollinearity?

Multicollinearity occurs when predictors in a regression model contain strongly overlapping linear information. Predictive accuracy may remain acceptable, but separating one adjusted coefficient from the others becomes difficult. Standard errors can increase, signs may change and small data perturbations can produce large coefficient shifts.

Using VIF

For predictor j, VIF=1/(1−R²), where R² comes from regressing that predictor on the remaining predictors. Thresholds such as 5 or 10 are not universal laws. Interpret VIF in relation to sample size, coefficient precision and the scientific role of the variable.

Useful remedies

• Clarify whether the goal is causal explanation or prediction.

• Combine redundant measures only when scientifically defensible.

• Do not remove an essential confounder merely because its VIF is high.

• Use ridge regression for stable prediction or carefully validated lasso selection.

• Consider principal components when prediction matters more than direct coefficient interpretation.

• Centre variables to help with polynomial or interaction terms; centring does not cure overlap between distinct measures.

Dummy-variable trap

Including all levels of a categorical predictor together with an intercept creates exact linear dependence. Leave a reference category or use an appropriate contrast coding scheme.

Reporting

Report diagnostics alongside coefficient stability and sensitivity analyses. Show how conclusions change under alternative variable definitions or penalised models rather than relying on one mechanical cutoff.

References

• O'Brien RM. A Caution Regarding Rules of Thumb for VIF. 2007.

• Tibshirani R. Regression Shrinkage and Selection via the Lasso. 1996.

Accessed 20 June 2026.