Data Analysis
Panel Data Analysis: Fixed Effects, Random Effects and the Hausman Test
GOSPELTRADER Research Desk · 12 September 2026 · 11 min read
Quick answer
Panel data follows the same units over time, so it can control for unobserved characteristics that do not change within a unit. Fixed effects uses only within-unit variation and removes all time-invariant confounders; random effects also uses between-unit variation and is more efficient, but assumes the unit effect is uncorrelated with the predictors. The Hausman test compares them: a significant result (p < .05) favours fixed effects.
1. What panel data buys you
A panel has two indices: unit i and time t. Because each unit is observed repeatedly, anything about that unit that is stable over the observation window — management culture, geography, innate ability — can be differenced away, even though it was never measured.
That is the core advantage over cross-sectional regression, and also the core limitation: fixed effects cannot estimate the effect of anything that does not change within a unit.
Pooled OLS Y_it = β₀ + βX_it + ε_it (ignores the panel structure)
Fixed effects Y_it = βX_it + α_i + ε_it (α_i estimated or differenced out)
Random effects Y_it = β₀ + βX_it + u_i + ε_it (u_i treated as random, E[u_i|X] = 0)
Within transformation (FE)
(Y_it - Ȳ_i) = β(X_it - X̄_i) + (ε_it - ε̄_i)2. Choosing the estimator
The choice is an identification decision, not a statistical convenience. State it explicitly in the write-up, along with the test that supports it.
| Estimator | Uses | Choose when | Cannot estimate |
|---|---|---|---|
| Pooled OLS | All variation | No meaningful unit effect (rejected by the Breusch-Pagan LM test) | — |
| Fixed effects | Within-unit variation | Unit effects correlate with predictors; causal inference is the goal | Time-invariant predictors |
| Random effects | Within + between | Unit effects are uncorrelated with predictors; time-invariant predictors matter | — |
| First differences | Change-on-change | Two periods, or serially correlated errors | Time-invariant predictors |
3. The specification tests, in order
Run these three in sequence and report all of them; each answers a different question and skipping one leaves the specification unjustified.
- • Breusch-Pagan LM test — pooled OLS versus random effects. Significant means a unit effect exists, so do not pool.
- • Hausman test — random versus fixed effects. Significant (p < .05) means the random-effects assumption fails; use fixed effects.
- • F-test on the unit dummies — pooled OLS versus fixed effects. Significant means the unit intercepts differ jointly.
- • Add time dummies and test them jointly; significant time effects mean a two-way fixed effects model.
Hausman statistic
H = (β_FE - β_RE)' [Var(β_FE) - Var(β_RE)]⁻¹ (β_FE - β_RE) ~ χ²(k)
Interpretation
p < .05 → estimators differ systematically → fixed effects
p ≥ .05 → random effects is consistent and more efficient4. Standard errors are where most panel papers fail
Default standard errors assume independent errors, which a panel almost never has: observations within a unit are correlated across time. Cluster the standard errors at the unit level as the default, and report that you did so.
With few clusters (under roughly 40), clustered errors are biased downward; use a wild cluster bootstrap instead. With correlation across units in the same period — common in macro and market panels — use Driscoll-Kraay standard errors.
- • Wooldridge test for serial correlation in panel errors — significant means cluster or model an AR(1) term.
- • Modified Wald test for groupwise heteroscedasticity — significant means robust standard errors are mandatory.
- • Pesaran CD test for cross-sectional dependence — significant means Driscoll-Kraay.
- • For long panels (large T), test for unit roots with Im-Pesaran-Shin or Levin-Lin-Chu before regressing levels on levels.
5. Reporting template
Report the estimator and why it was chosen, the number of units and periods, whether the panel is balanced, the clustering level, within/between/overall R², and the coefficients with clustered standard errors.
Example: "We estimate a two-way fixed effects model on a balanced panel of 120 firms observed quarterly from 2019 to 2024 (N = 2,880). The Hausman test rejected random effects, χ²(5) = 21.44, p < .001. Standard errors are clustered at the firm level. Within R² = .34."
Frequently asked questions
What is the difference between panel data and repeated cross-sections?
A panel follows the same units over time, so within-unit change can be measured. Repeated cross-sections sample different units each period, so only aggregate change can be studied and fixed effects at the unit level are impossible.
Can fixed effects estimate the effect of gender or country?
No. The within transformation removes anything constant within a unit, so time-invariant predictors drop out. If such variables are central, use random effects (after justifying the assumption), a correlated random effects specification, or interact the time-invariant variable with time.
What if my panel is unbalanced?
Unbalanced panels are fine and standard software handles them. The concern is why observations are missing: if units drop out for reasons related to the outcome, the attrition is non-random and estimates are biased. Test for selective attrition and discuss it.
How many time periods do I need?
Fixed effects works with as few as two periods, though precision is poor. Meaningful within-unit variation usually needs three or more. Long panels (T above roughly 20) raise separate concerns — unit roots, cross-sectional dependence — that short panels can ignore.
Want this analysis done for you?
We deliver cleaning, assumption testing, modelling and a reporting-ready write-up.