=Calculator
How long does your
test have to run?
The same calculation abdriven runs before every launch. Enter your numbers before you build the variation, not once the result already looks exciting.
How it is calculated
- The basis is a two-sided test with 80% power, the standard abdriven plans with too.
- More than two variations raise the required sample, because the level is corrected for the extra comparisons.
- The runtime is rounded up to at least 14 days, so every weekday occurs twice.
- The calculation uses the visitors who actually see the experiment, not your whole site’s traffic.
FAQ
Sample size and runtime calculator
What should I enter as the smallest difference?
The smallest gain that would actually make you ship the change. Enter 2% and you get a runtime of months; enter 30% and you will only find very coarse effects.
And if the runtime comes out too long?
Then test something bigger, or test on a page with more traffic. Simply stopping a too-small test early is not a solution, it is the road to a false winner.
Calculators
One more calculator.
Make your next change an informed one
Less guesswork.
More now we know.
One question is a good place to start.