A/B Test Significance Calculator
A/B Test Significance Calculator is a free online tool on MaticsBox. Use it directly in your browser without installing additional software.
Checks whether the difference in conversion rate between two A/B test variants is statistically significant, using a standard two-proportion z-test at 95% confidence. Helps avoid calling a winner based on a difference that could just be random noise.
Enter or paste your data into the form, adjust the available options if needed, and run the tool. The result will be shown on the same page.
FAQ
What does "statistically significant" mean here?
It means the observed difference between variants is unlikely (less than 5% probability) to have happened by chance alone, based on the sample sizes and conversion counts entered.
How many visitors/conversions do I need for a reliable result?
There is no fixed minimum, but very small sample sizes (a few dozen visitors) rarely produce a reliable significant result even with a real underlying difference — bigger samples give more trustworthy conclusions.
Does significance mean I should definitely pick the winning variant?
It's strong evidence, but also consider practical significance (is the improvement big enough to matter) and whether the test ran long enough to capture normal variation (e.g. day-of-week effects).