This free calculator answers the two questions every A/B test needs: how many visitors do I need, and is my result real. Use the first part before you start a test and the second part after it has run its planned length.

All calculations run in your browser. Nothing you enter is sent anywhere.

Laptop showing analytics charts on a desk

1. How many visitors do I need?

Estimate the sample size per version before you start a test.

2. Is my result significant?

Compare two versions after the test has run its planned length.

How to use the sample size calculator

  1. Current conversion rate: the rate of the page or step you are testing today, for example 3% of visitors who submit a form.
  2. Rate you hope to detect: the smallest improvement worth acting on. A change from 3% to 3.6% is a 20% relative lift.
  3. Confidence: how sure you want to be that a winner is real. 95% is the usual choice.
  4. Power: the chance of detecting the improvement if it really exists. 80% is standard.
  5. Visitors per week: optional; the calculator estimates how many weeks the test needs.

If the answer is many months, the test is not realistic. Test a bigger change, test on a higher-traffic page, or improve the page using research instead. The guide to A/B testing on low-traffic sites explains these options.

How to read the significance result

  • Significant: the difference is unlikely to be chance at your chosen confidence. Check that the test ran its full planned length and that nothing unusual (a sale, an outage, a festival spike) affected one version.
  • Not significant: you cannot tell the versions apart with this data. Treat it as a tie, keep the simpler version, and move on to a bolder test.
  • p-value: the probability of seeing a difference at least this large if the versions were truly the same. Lower is stronger evidence.
  • Relative change: how much B differs from A as a percentage of A’s rate.

The calculator uses a standard two-proportion z-test. For tests with very few conversions (fewer than about 10 in either version), results are unreliable whatever the calculator says.

Rules that prevent false winners

  • Decide sample size and duration first, then do not stop early.
  • Run full weeks. Weekday and weekend behaviour differ, and Nepali festivals change behaviour sharply.
  • Test one main change at a time unless you have the traffic for multivariate tests.
  • Measure the conversion that matters: orders, qualified leads, or bookings, not just clicks.
  • Record every test with the hypothesis, dates, result, and decision.

Hypothetical example: a Kathmandu online store converts 3% of visitors and wants to detect a lift to 3.6%. The calculator says about 13,900 visitors per version. At 4,000 visitors a week, that is about seven weeks. The team instead tests a bigger change (a simplified checkout) that they expect to lift conversion to 4.5%, which needs about 2,500 visitors per version, under two weeks.