Type I errors are false positives (rejecting a true null hypothesis) and Type II errors are false negatives (missing a real effect).
Card 1 of 259. Answer: Type I errors are false positives (rejecting a true null hypothesis) and Type II errors are false negatives (missing a real effect).
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
A Type I error is rejecting a true null hypothesis, a false positive. A Type II error is failing to reject a false null hypothesis, a false negative. Their probabilities are α, the significance level, and β; 1 − β is the test's power.
Example: an A/B test of a new checkout page uses α = 0.05 and 80% power to detect a 1-point lift. If the page changes nothing, there is still a 5% chance of declaring a winner, a Type I error. If it adds 1 point, there is a 20% chance of missing it, a Type II error. Across 100 tests of useless changes, about 5 will look significant by luck.
With a fixed sample size, lowering α raises β; reducing both takes more data or a larger effect. Which error is worse depends on the costs: if the null is "the defendant is innocent", convicting an innocent person is a Type I error. Jerzy Neyman and Egon Pearson formalized both errors in papers from 1928 on.
