Inferential Statistics
Inferential statistics is the bridge from data in hand to claims about the world. This area covers the philosophy of hypothesis testing, the construction and interpretation of confidence intervals, and the full catalogue of common tests used in the life sciences and clinical research.
Tutorials are grouped by the kind of research question being answered: comparing a single mean to a reference, comparing two or more groups, testing associations between categorical variables, and quantifying correlations between continuous variables. Non-parametric alternatives are treated in parallel with their parametric counterparts, and multiple testing correction is given its own dedicated thread.
What is covered
- The logic of Null Hypothesis Significance Testing, type I/II errors, and the much-misunderstood p-value
- Confidence intervals: construction, coverage, and why they are more informative than p-values alone
- One-sample and paired-sample t-tests with assumption checks
- Two-sample t-tests: Student’s, Welch’s, and when to choose which
- One-way, two-way, repeated-measures, and mixed ANOVA with post-hoc testing
- Chi-squared tests of independence and goodness-of-fit; Fisher’s exact test for small tables
- Correlation coefficients: Pearson, Spearman, Kendall, and their inferential procedures
- Non-parametric rank tests: Mann-Whitney U, Wilcoxon signed-rank, Kruskal-Wallis, Friedman
- Permutation and bootstrap tests for when distributional assumptions are untenable
- Multiple testing correction: Bonferroni, Holm, Benjamini-Hochberg (FDR), Benjamini-Yekutieli
Effect sizes are reported alongside p-values in every example, and guidance on effect-size interpretation is given for each design.
Tutorials
TUTORIAL
Anderson-Darling Test
A distribution goodness-of-fit test with emphasis on the tails
TUTORIAL
Bartlett’s Test
Test of equal variances across groups, most powerful under normality but very sensitive to its violation
TUTORIAL
Benjamini-Hochberg FDR
Controlling the expected proportion of false discoveries among rejected hypotheses
TUTORIAL
Benjamini-Yekutieli FDR
Dependence-robust false discovery rate control at the cost of conservatism
TUTORIAL
Bonferroni Correction
Controlling family-wise error rate by dividing alpha across m tests
TUTORIAL
Bootstrap Confidence Intervals
Percentile, basic, BCa, and studentised bootstrap CIs
TUTORIAL
Chi-Squared Goodness-of-Fit Test
Testing whether observed category frequencies match an expected distribution
TUTORIAL
Chi-Squared Test of Independence
Testing association between two categorical variables via observed vs. expected counts under independence
TUTORIAL
Cochran-Mantel-Haenszel Test
Stratified analysis of 2x2 tables across levels of a confounding variable
TUTORIAL
Effect Sizes: Overview
Why effect sizes matter and which measures apply to which tests
TUTORIAL
Equivalence Testing with TOST
Two one-sided tests for establishing practical equivalence within a margin
TUTORIAL
Fisher’s Exact Test
Exact test for 2x2 (and r x c) contingency tables, based on the hypergeometric distribution
TUTORIAL
Friedman Test
Non-parametric test for three or more paired / repeated measures
TUTORIAL
Holm’s Step-Down Correction
Sequential Bonferroni with uniformly greater power; controls FWER
TUTORIAL
Kendall’s Tau
Rank correlation based on concordant vs discordant pairs; robust to ties and small samples
TUTORIAL
Kolmogorov-Smirnov Test
CDF-based goodness-of-fit for one-sample and two-sample comparisons
TUTORIAL
Kruskal-Wallis Test
Non-parametric one-way ANOVA based on ranks across three or more groups
TUTORIAL
Levene’s Test of Variances
Robust test of variance homogeneity across groups
TUTORIAL
Mann-Whitney U Test
Non-parametric two-sample test based on ranks; alternative to the independent t-test
TUTORIAL
McNemar’s Test
Paired / matched binary data: comparing two measurements on the same units, using discordant pairs only
TUTORIAL
Mixed ANOVA
Designs combining between-subjects and within-subjects factors, with the interaction as key
TUTORIAL
Multiple Comparisons: Overview
Family-wise error vs false discovery rate and when each applies
TUTORIAL
Null and Alternative Hypotheses
Formulating H0 and H1, choosing one- vs two-sided tests, and avoiding post-hoc reformulation
TUTORIAL
One-Proportion Test
Testing whether an observed proportion differs from a pre-specified reference; exact and score-based methods
TUTORIAL
One-Sample t-Test
Testing whether a sample mean equals a pre-specified reference value
TUTORIAL
One-Sample z-Test
Comparing a sample mean to a reference when the population variance is known – rare in practice but pedagogically useful
TUTORIAL
One-Way ANOVA
Comparing means across three or more independent groups via the F-test
TUTORIAL
P-Values Explained
Definition, correct interpretation, and the most common misinterpretations of the p-value
TUTORIAL
Paired t-Test
Comparing two dependent measurements by applying a one-sample t-test to their differences
TUTORIAL
Pearson Correlation Test
Testing whether the Pearson correlation between two continuous variables is non-zero
TUTORIAL
Permutation Tests
Exact or Monte Carlo p-values via resampling under exchangeability
TUTORIAL
Post-Hoc Tests with Tukey HSD
Pairwise comparisons after ANOVA with family-wise error control
TUTORIAL
Repeated-Measures ANOVA
Within-subjects analysis of variance with sphericity checks and corrections
TUTORIAL
Shapiro-Wilk Normality Test
The most powerful commonly used test for normality in small to moderate samples
TUTORIAL
Spearman Rank Correlation Test
Non-parametric correlation test for monotonic association between two variables
TUTORIAL
The Bootstrap: Introduction
Resampling with replacement to estimate standard errors and sampling distributions
TUTORIAL
The Jackknife
Leave-one-out resampling for bias and variance estimation
TUTORIAL
The Runs Test
Testing randomness of a binary or dichotomised sequence via run lengths
TUTORIAL
The Sign Test
Testing a median or paired difference using only the direction of each observation
TUTORIAL
Two-Proportion Test
Comparing two independent proportions; z-test and chi-squared equivalence, with risk difference and relative risk
TUTORIAL
Two-Sample t-Test
Comparing means between two independent groups with Student’s and Welch’s t-tests, including assumption checks and effect sizes
TUTORIAL
Two-Way ANOVA
Two between-subjects factors: main effects and interaction
TUTORIAL
Type I and Type II Errors
Rejecting a true null (alpha) and failing to reject a false null (beta); power and the trade-off
TUTORIAL
Welch’s t-Test
Two-sample t-test that does not assume equal variances; R’s default
TUTORIAL
Wilcoxon Signed-Rank Test
Non-parametric paired-sample test based on signed ranks