Data Analysis
Regression Analysis: Assumptions, Interpretation and Worked Examples
GOSPELTRADER Research Desk · 11 September 2026 · 13 min read
Quick answer
Regression analysis estimates how much a continuous outcome changes per unit change in one or more predictors, holding the others constant. Its validity depends on five assumptions — linearity, independence, homoscedasticity, normally distributed errors and absence of severe multicollinearity — each of which must be tested and reported alongside β, R² and the F-test.
The model and how it is estimated
Multiple linear regression models the conditional mean of Y as a linear function of predictors. Ordinary Least Squares chooses the coefficients that minimise the sum of squared residuals.
Population model
Y = β₀ + β₁X₁ + β₂X₂ + ... + βₖXₖ + ε , ε ~ N(0, σ²)
OLS criterion
minimise Σ (yᵢ - ŷᵢ)²
β̂ = (XᵀX)⁻¹ Xᵀ y
Simple regression slope
β₁ = r * (s_y / s_x) and β₀ = ȳ - β₁x̄
Fit
R² = SSR/SST = 1 - SSE/SST
Adjusted R² = 1 - (1 - R²)(n - 1)/(n - k - 1)
F = (SSR/k) / (SSE/(n - k - 1))The five assumptions and how to test each
Report these before the coefficients. Every one has a specific diagnostic and a specific remedy.
| Assumption | Diagnostic | Remedy if violated |
|---|---|---|
| Linearity | Residuals vs fitted plot; partial plots | Add polynomial or log term; spline |
| Independence of errors | Durbin-Watson (1.5-2.5) | Time-series model; cluster-robust SE |
| Homoscedasticity | Breusch-Pagan; residual funnel shape | Robust (HC3) SE; log-transform Y |
| Normality of residuals | Q-Q plot; Shapiro-Wilk on residuals | Transform Y; bootstrap CIs |
| No severe multicollinearity | VIF < 5-10; tolerance > 0.10 | Drop or combine predictors; ridge |
| No influential outliers | Cook's D > 1; |std residual| > 3.29 | Investigate, justify, report both models |
Worked example: predicting monthly revenue
A retailer models monthly revenue (₦'000) on advertising spend (₦'000) and number of active outlets, n = 120.
Estimated model
Revenue = 412.6 + 3.18 (AdSpend) + 96.4 (Outlets)
AdSpend : B = 3.18, SE = 0.42, β = .52, t = 7.57, p < .001, 95% CI [2.35, 4.01]
Outlets : B = 96.4, SE = 21.8, β = .31, t = 4.42, p < .001, 95% CI [53.2, 139.6]
R² = .58, Adj. R² = .57, F(2, 117) = 80.7, p < .001
Interpretation
Each additional ₦1,000 of advertising is associated with a ₦3,180 increase
in monthly revenue, holding outlet count constant.
The model explains 58% of the variance in revenue.
Advertising (β = .52) has the larger standardised effect.Unstandardised vs standardised coefficients
B answers 'how much does Y change per one unit of X in real units' — use it for practical decisions and forecasting. β (beta) answers 'which predictor matters more' by putting all predictors on a common standard-deviation scale — use it for relative importance. Never compare raw B values measured in different units and call one stronger.
- • B: interpret in the measurement units of the data, with its confidence interval.
- • β: unit-free, comparable within the same model only.
- • R²: proportion of variance explained; adjusted R² penalises useless predictors.
- • Statistical significance is not effect size — with n = 5,000 a trivial β becomes significant.
Common errors that invalidate a regression chapter
These are the issues that most often force revisions.
- • Using linear regression on a binary or count outcome — use logistic or Poisson regression instead.
- • Interpreting association as causation without a design that supports it.
- • Ignoring omitted variable bias: a missing confounder biases every coefficient in the model.
- • Stepwise selection driven by p-values, which capitalises on chance; select predictors from theory.
- • Extrapolating beyond the observed range of the predictors.
- • Reporting R² alone with no assumption diagnostics or confidence intervals.
Choosing the right regression family
The outcome variable dictates the model.
| Outcome type | Model | Coefficient reported as |
|---|---|---|
| Continuous | OLS linear regression | B / β |
| Binary (yes/no) | Binary logistic regression | Odds ratio, Exp(B) |
| Ordered categories | Ordinal (proportional odds) regression | Odds ratio |
| Unordered categories | Multinomial logistic regression | Relative risk ratio |
| Counts | Poisson or negative binomial | Incidence rate ratio |
| Time to event | Cox proportional hazards | Hazard ratio |
| Repeated / nested data | Mixed-effects (multilevel) model | Fixed effects + variance components |
Frequently asked questions
What is a good R² value?
It depends entirely on the field. In physical measurement, R² of .90 is expected; in social and behavioural research, .20 to .40 is common and publishable because human behaviour has many unmeasured causes. Judge a model by theoretical relevance, assumption compliance and effect sizes rather than by R² alone.
How many predictors can I include?
Use n ≥ 50 + 8m as a working minimum for the overall model and n ≥ 104 + m for testing individual predictors, where m is the number of predictors. Overloading a small sample produces unstable coefficients that will not replicate.
What VIF value indicates a multicollinearity problem?
VIF above 10 signals a serious problem, and many methodologists act at 5. Remedies include removing one of the correlated predictors, combining them into a composite index, centring interaction terms, or using ridge regression.
Can regression prove causation?
No. Regression estimates conditional association. Causal claims require a design that rules out confounding — randomised assignment, difference-in-differences, instrumental variables, regression discontinuity or a fixed-effects panel — plus explicit assumptions stated in the write-up.
Want this analysis done for you?
We deliver cleaning, assumption testing, modelling and a reporting-ready write-up.