Proportion Z-Tests for People Who Don’t Live in Stats Class

You launched a change. Conversion went from 4.1% to 4.6%. Someone asks if it’s “significant.” That’s not a vibe check; it’s a proportion test—if the data is clean.

A one-proportion z-test asks whether an observed rate differs from a hypothesized rate (or, in related setups, whether two rates differ). The 1-proportion z-test calculator handles the arithmetic. You still own the design.

Before you click calculate

• Were units independent? The same user refreshing five times is not five users.
• Is n large enough? Rules of thumb about np and n(1-p) exist for a reason.
• Did you peek daily and stop when the chart looked good? That’s not a planned test.

I’ve seen “significant” lifts vanish when bot traffic was filtered. The math was fine. The denominator was garbage.

How I use the output

If the p-value is tiny but the effect is 0.05 percentage points on a low-traffic page, I still might not ship a complex feature. Statistical significance isn’t business importance. Conversely, a noisy non-significant result on a short test isn’t proof of zero effect—it might mean “come back with more data.”

Write the hypothesis in one sentence before you open the calculator. If you can’t, you’re fishing.