Extensions of Multiple Regression

Real data contain unusual observations, qualitative factors and yes-or-no outcomes. This module shows how to detect observations that drive a regression (leverage, studentized residuals and Cook's distance), how to use dummy variables to let intercepts and slopes differ between groups, and how to model binary outcomes such as default with logistic regression, interpreting odds and testing variables with the likelihood ratio test.

Start this lesson free Practice questions

What you will learn

1. Detect influential observations using leverage, studentized residuals and Cook's distance.

High leverage = extreme X values; flag h_ii above 3(k + 1)/n.
Outlier = extreme Y given X; use the studentized residual, compared with t at n - k - 2 df.
Cook's D combines residual size and leverage; above 0.5 possibly influential, above 1 likely, or above 2 times the square root of k/n.
A high-leverage point that lies on the regression line has little influence.
Investigate flagged points (data errors, different population) and report results with and without them.

2. Build and interpret regressions that include dummy variables for intercept and slope differences.

A dummy takes 1 when a condition holds and 0 otherwise; the intercept belongs to the reference group.
An intercept dummy shifts the level; a slope dummy (D x X) changes the sensitivity to X.
With both, the D = 1 group has intercept b0 + d0 and slope b1 + d1; the gap at X is d0 + d1X.
Use n - 1 dummies for n categories; using n with an intercept is the dummy variable trap.
t-tests on dummy coefficients test level or slope differences; a joint F-test checks whether one line fits both groups.

3. Estimate and interpret a logistic regression for a binary outcome, including odds and the likelihood ratio test.

Logit models the log odds, ln[P/(1 - P)], as linear in the X variables, keeping predicted probabilities between 0 and 1.
A coefficient is the change in log odds per unit of X; e to the power of the coefficient is the odds ratio.
P = 1/(1 + e^(-L)), where L is the linear predictor; the probability effect of X varies along the S-curve.
Estimation is by maximum likelihood; there is no ordinary R-squared, only a pseudo R-squared.
LR = -2(ln L_R - ln L_U), chi-square with q df, tests a group of variables jointly.

Module at a glance

Leverage measures extreme X values (flag above 3(k + 1)/n); studentized residuals flag extreme Y values (t with n - k - 2 df).
Cook's D combines residual and leverage; above 0.5 possibly and above 1 likely influential, or above 2 times the square root of k/n.
Intercept dummies shift the level; slope dummies (D x X) change the sensitivity; the group gap at X is d0 + d1X.
Use n - 1 dummies for n categories when the model has an intercept.
Logistic regression models log odds; P = 1/(1 + e^(-L)); e to the coefficient is the odds ratio.
Logit models are estimated by maximum likelihood; LR = -2(ln L_R - ln L_U) tests groups of variables (chi-square, q df).

This free module has 3 lessons, 14 practice questions with full explanations.

Start the lesson