How to Use the A/B Test Significance & Sample Size Calculator
-
1
Mode 1 (Results Significance): Enter visitors and conversions for Control (A) and Variation (B).
-
2
The calculator computes conversion rate uplift %, Z-score, p-value, and declares whether the result is statistically significant at 95% or 99% confidence.
-
3
Mode 2 (Sample Size Pre-test): Enter baseline conversion rate and Minimum Detectable Effect (MDE) to compute required sample size per variation before starting your test.
Practical Example & Use Case
Common Real-World Scenario
Variation B with 120 conversions from 2,000 visitors (6.0%) vs Control with 90 conversions from 2,000 visitors (4.5%) achieves 95.8% statistical significance (p = 0.042).
Technical Notes & Limitations
Browser Engine Architecture
Uses two-tailed two-proportion Z-test statistical distribution modeling.
Frequently Asked Questions
Why is 95% statistical confidence the standard?
A 95% confidence level means there is only a 5% chance that the observed difference is due to random variance rather than the change itself.