Evaluating Regression Model Fit and Interpreting Model Results

An estimated regression is only useful if it fits well enough, its coefficients are reliably different from zero and its forecasts come with honest error bands. This module shows how to read the ANOVA table, compare models with adjusted R-squared and information criteria, run t-tests and joint F-tests, and produce forecasts while recognising every source of uncertainty. Level II item sets routinely supply partial regression output and ask candidates to complete and interpret it.

Start this lesson free Practice questions

What you will learn

1. Read an ANOVA table and judge model fit using R-squared, adjusted R-squared, the standard error of the estimate and information criteria.

SST = SSR + SSE; degrees of freedom are k, n - k - 1 and n - 1; MSE = SSE/(n - k - 1).
R-squared never falls when a variable is added; adjusted R-squared rises only if the new variable's |t| exceeds 1.
SEE = square root of MSE, the typical residual size in the units of Y.
AIC (penalty 2 per parameter) suits prediction; BIC (penalty ln n per parameter) is stricter and suits goodness of fit.
Lower AIC or BIC is better; compare only models estimated on the same sample.

2. Test whether a single coefficient, or a group of coefficients jointly, differs from zero using t- and F-tests.

t = (estimate - hypothesised value) / standard error, with n - k - 1 degrees of freedom.
Hypothesised values other than zero (for example beta = 1) use the same t-test.
Joint F = [(SSE_R - SSE_U)/q] / [SSE_U/(n - k - 1)], with q and n - k - 1 degrees of freedom; one-tailed.
The overall F-test (all slopes zero) is MSR/MSE with k and n - k - 1 degrees of freedom.
Correlated variables can be jointly significant even when no single one is.

3. Produce a forecast of the dependent variable from an estimated equation and explain the sources of forecast uncertainty.

Forecast = intercept + sum of each coefficient times the forecast value of its variable.
Use every variable in the estimated model; re-estimate first if a variable is to be removed.
Uncertainty comes from the error term (SEE), sampling error in the coefficients, and errors in forecasting the inputs.
Sampling error matters most when the inputs are far from their sample means.
The standard error of a forecast exceeds the SEE, so an interval based on the SEE alone is too narrow.

Module at a glance

SST = SSR + SSE; R-squared = SSR/SST; MSE = SSE/(n - k - 1); SEE = square root of MSE.
Adjusted R-squared penalises extra variables and rises only when a new variable's |t| exceeds 1.
Lower AIC is preferred for prediction, lower BIC (heavier penalty) for parsimonious fit.
t = (estimate - hypothesised value) / standard error with n - k - 1 df; the hypothesised value need not be zero.
Joint F = [(SSE_R - SSE_U)/q] / [SSE_U/(n - k - 1)]; the overall F-test is MSR/MSE.
Forecast with every variable in the estimated model, including insignificant ones.
Forecast uncertainty combines model error, coefficient sampling error and errors in the inputs, so intervals exceed plus or minus t times SEE.

This free module has 3 lessons, 13 practice questions with full explanations.

Start the lesson