Hypothesis Testing
You now know how to summarise data (Notebook 38) and model randomness (Notebook 39). Hypothesis testing puts them together to answer the question every analyst is eventually asked: "Is this difference real, or could it just be chance?" We'll build the logic from scratch, run the workhorse tests with scipy.stats, and — just as importantly — learn the traps (p-hacking, misreading p-values, ignoring effect size) that make hypothesis testing the most misused tool in statistics.
Notebook
About 180 minutes to complete
Enrol to read the notebook and download it to run.
What this module covers
You now know how to summarise data (Notebook 38) and model randomness (Notebook 39). Hypothesis testing puts them together to answer the question every analyst is eventually asked: "Is this difference real, or could it just be chance?" We'll build the logic from scratch, run the workhorse tests with scipy.stats, and — just as importantly — learn the traps (p-hacking, misreading p-values, ignoring effect size) that make hypothesis testing the most misused tool in statistics.
By the end of it
- State the logic of hypothesis testing — null vs alternative, test statistic, p-value, significance level α
- Distinguish Type I vs Type II errors and understand statistical power
- Run and interpret t-tests: one-sample, two-sample (independent, incl. Welch's), and paired
- Run chi-square tests: goodness-of-fit and test of independence
- Report effect sizes (Cohen's d, Cramér's V) and separate statistical from practical significance
- Check test assumptions (normality, equal variance) and choose the right test
- Recognise and avoid p-hacking, multiple-comparison inflation, and common p-value misinterpretations