A/B Testing Equations

Where N is the number of trials, c is the current conversion date, and d is the effect size, i.e. the observed new variant conversion rate minus the current conversion rate, you can compute how many trials N you need to run to have a valid test that has not been "peeked," i.e. not stopped too early:

In a spreadsheet like Excel or Google Sheets, substituting those numbers with the right cell references:

=16*(c-c^2)/d^2

---

Solving the equation for d, we answer the question: Given this many trials N, and the baseline conversion rate c, what is the smallest effect size d that we can detect with statistical accuracy?

In a spreadsheet like Excel or Google Sheets, substituting those numbers with the right cell references:

=4*SQRT((c-c^2)/N)