A hypothesis test whose statistic follows a standard normal distribution under the null hypothesis: the difference divided by its standard error, compared with ±1.96 for a two-sided test at 5%. It applies when the standard error is known or the sample is large, as with conversion rates.
Card 1 of 276. Answer: A hypothesis test whose statistic follows a standard normal distribution under the null hypothesis: the difference divided by its standard error, compared with ±1.96 for a two-sided test at 5%. It applies when the standard error is known or the sample is large, as with conversion rates.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
The z-test and the t-test share the same statistic, an estimate minus its null value over a standard error. The difference is the reference distribution: the z-test uses the normal, which is right when the population standard deviation is known or the sample is large enough that estimating it adds little error. For proportions the standard error follows from the proportion itself, so the two-proportion z-test is the usual test for A/B conversion rates.
Example: variant A converts 200 of 2,000 users (10%) and variant B converts 240 of 2,000 (12%). The pooled rate is 440 / 4,000 = 11%, so the standard error of the difference is √(0.11 × 0.89 × (1/2,000 + 1/2,000)) ≈ 0.0099. Then z = 0.02 / 0.0099 ≈ 2.02 and the two-sided p ≈ 0.043.
Follow-ups: with a small sample and an estimated standard deviation, use the t-test, whose heavier tails account for the extra uncertainty; above a few dozen observations the two give almost the same p-value. For a 2 × 2 table, z² equals the chi-square statistic without continuity correction, here 4.09, so the two tests agree.
