4. Regression & Statistical Modelling in R
Statistical models let you quantify relationships between variables, make predictions, and test hypotheses about causal mechanisms. This tutorial covers the modelling workflow from simple linear regression to mixed-effects models.
Setup Link to heading
library(tidyverse)
library(broom)
data(mtcars)
Simple Linear Regression Link to heading
# Does weight predict MPG?
model1 <- lm(mpg ~ wt, data = mtcars)
summary(model1)
Interpreting the Output Link to heading
- Intercept (β₀): Expected MPG when weight = 0 (extrapolation)
- Slope (β₁): For each 1000 lb increase in weight, MPG changes by β₁
- R²: Proportion of variance in MPG explained by weight
- p-value: Probability of observing this slope if the true slope were 0
Tidy Output with broom
Link to heading
tidy(model1) # Coefficient table
glance(model1) # Model-level statistics
augment(model1) # Observation-level statistics (fitted, residuals)
Multiple Regression Link to heading
model2 <- lm(mpg ~ wt + hp + cyl, data = mtcars)
summary(model2)
# Compare models
anova(model1, model2)
AIC(model1, model2)
BIC(model1, model2)
Multicollinearity Link to heading
library(car)
# Variance Inflation Factors
vif(model2)
# VIF > 5 suggests problematic collinearity
Model Diagnostics Link to heading
# Diagnostic plots
par(mfrow = c(2, 2))
plot(model2)
What to look for: Link to heading
- Residuals vs Fitted: Should show no pattern (linearity)
- Q-Q Plot: Points should follow the diagonal (normality)
- Scale-Location: Horizontal band (homoscedasticity)
- Residuals vs Leverage: Identify influential observations
# Formal tests
# Normality of residuals
shapiro.test(residuals(model2))
# Homoscedasticity (Breusch-Pagan)
library(lmtest)
bptest(model2)
# Autocorrelation (Durbin-Watson)
dwtest(model2)
Interaction Effects Link to heading
model3 <- lm(mpg ~ wt * factor(cyl), data = mtcars)
summary(model3)
# Visualise interaction
ggplot(mtcars, aes(x = wt, y = mpg, colour = factor(cyl))) +
geom_point(size = 3) +
geom_smooth(method = "lm", se = TRUE) +
labs(title = "Weight × Cylinders Interaction",
colour = "Cylinders") +
theme_minimal()
Polynomial Regression Link to heading
model_poly <- lm(mpg ~ poly(wt, 2), data = mtcars)
summary(model_poly)
ggplot(mtcars, aes(x = wt, y = mpg)) +
geom_point() +
geom_smooth(method = "lm", formula = y ~ poly(x, 2), colour = "red") +
labs(title = "Quadratic Fit: Weight vs MPG") +
theme_minimal()
Generalised Linear Models (GLM) Link to heading
Logistic Regression Link to heading
# Predict transmission type (0 = automatic, 1 = manual)
model_logit <- glm(am ~ wt + hp, data = mtcars, family = binomial)
summary(model_logit)
# Odds ratios
exp(coef(model_logit))
exp(confint(model_logit))
Poisson Regression Link to heading
# Count data example
model_pois <- glm(carb ~ wt + hp, data = mtcars, family = poisson)
summary(model_pois)
# Check for overdispersion
deviance(model_pois) / df.residual(model_pois)
# Values >> 1 indicate overdispersion
Mixed-Effects Models Link to heading
When data has a hierarchical or grouped structure (e.g., students within schools, measurements within subjects):
library(lme4)
# Using sleepstudy dataset (reaction time over days of sleep deprivation)
data(sleepstudy)
# Random intercept model
mixed1 <- lmer(Reaction ~ Days + (1 | Subject), data = sleepstudy)
summary(mixed1)
# Random intercept and slope
mixed2 <- lmer(Reaction ~ Days + (Days | Subject), data = sleepstudy)
summary(mixed2)
# Compare models
anova(mixed1, mixed2)
Visualising Random Effects Link to heading
# Plot individual trajectories
ggplot(sleepstudy, aes(x = Days, y = Reaction, group = Subject)) +
geom_line(alpha = 0.3) +
geom_smooth(aes(group = 1), method = "lm", colour = "red", linewidth = 1.5) +
labs(title = "Reaction Time Over Sleep Deprivation Days",
subtitle = "Grey = individual subjects, Red = population average") +
theme_minimal()
# Extract and plot random effects
ranef_df <- as.data.frame(ranef(mixed2)$Subject)
ranef_df$Subject <- rownames(ranef_df)
ggplot(ranef_df, aes(x = `(Intercept)`, y = Days)) +
geom_point() +
geom_hline(yintercept = 0, linetype = "dashed") +
geom_vline(xintercept = 0, linetype = "dashed") +
labs(title = "Random Effects: Intercept vs Slope",
x = "Random Intercept", y = "Random Slope") +
theme_minimal()
Model Selection Workflow Link to heading
# Step 1: Full model
full_model <- lm(mpg ~ wt + hp + disp + drat + qsec, data = mtcars)
# Step 2: Stepwise selection (use with caution)
step_model <- step(full_model, direction = "both")
summary(step_model)
# Step 3: Compare with AIC/BIC
AIC(model1, model2, full_model, step_model)
Reporting Results Link to heading
When reporting regression results:
- Unstandardised coefficients (β) with 95% CIs
- p-values for each predictor
- R² (or pseudo-R² for GLMs)
- F-statistic and its p-value (for overall model significance)
- Sample size (N)
# Create a publication-ready table
library(broom)
model2 %>%
tidy(conf.int = TRUE) %>%
mutate(across(where(is.numeric), ~ round(., 3)))
Key Takeaways Link to heading
- Always check model assumptions with diagnostic plots.
- Use
broomfor tidy model output. - Choose GLMs when the response variable is not continuous/normal.
- Use mixed models for grouped/hierarchical data.
- Report effect sizes and confidence intervals, not just p-values.
Previous: ← Inferential Statistics