Calculate required sample size for statistical tests. Part of the DevTools Surf developer suite. Browse more tools in the Statistics collection.
Use Cases
Calculate the minimum sample size needed for an A/B test at specified significance and power levels.
Determine whether a dataset already collected is large enough to detect a meaningful effect.
Plan clinical trial enrollment to achieve statistical power for a target effect size.
Scope a survey sample size for a market research study with a specified margin of error.
Tips
Power (1 - beta) should be set to 0.80 minimum for most research; 0.90 is preferred for clinical studies where missing a real effect has consequences.
Use the smallest expected effect size from prior literature, not the largest — over-optimistic effect size estimates lead to underpowered studies.
For A/B tests, account for multiple comparisons: if testing 5 variants, apply Bonferroni correction and divide alpha by 5.
Fun Facts
Statistical power analysis was largely formalized by Jacob Cohen in his 1969 book 'Statistical Power Analysis for the Behavioral Sciences,' which remains one of the most cited statistics texts ever written.
Most published psychology studies between 1960 and 2000 were powered at only 35–50%, meaning they had a coin-flip chance of detecting real medium-sized effects — a major contributor to the replication crisis.
A/B test sample size requirements exploded in the 2010s as tech companies ran thousands of simultaneous experiments; Google reportedly runs over 10,000 experiments per year on Search alone.
FAQ
What is statistical power?
Power (1 - beta) is the probability of detecting a real effect when it exists. Standard convention is 0.80 (80% chance of detection). Low power means a real effect may be missed (Type II error).
What effect size should I use?
Cohen's d for comparing two means: small=0.2, medium=0.5, large=0.8. For proportions, Cohen's h is appropriate. Use the smallest effect size that would be practically meaningful for your decision.
