New in the kilnstl → gcodemp3 → midipsd → pngpdf → csvutm builder
A/B Test Significance Calculator
Check whether your A/B test result is statistically significant: conversion rates, confidence intervals, z-score, p-value, and a plain verdict. Runs in your browser.
Private by construction. The conversion happens in this page's JavaScript. Nothing you paste here leaves your machine, and the tool keeps working with the network unplugged.
Paste your control on the first line and each variant below it — name, visitors, conversions — and get the analysis marketers usually pay a tool for: conversion rates with 95% confidence intervals, relative lift, z-score, two-tailed p-value, and a verdict in plain words. The math is the standard two-proportion pooled z-test, computed in your browser, so unreleased experiment numbers stay on your machine. More than two lines work too: each variant is tested against the control, which is the honest way to read a multi-arm test (testing every arm against every other arm inflates false positives).
Questions this tool gets asked
What test does this run?
A two-proportion z-test with a pooled standard error, two-tailed - the standard test for comparing conversion rates between two groups, and the same one most commercial A/B calculators run under the hood.
What does the p-value actually mean?
The probability of seeing a difference at least this large if the variants truly performed identically. Below 0.05, convention calls the result significant at 95% confidence. It is not the probability that B beats A - no frequentist test gives you that.
Why does my winning variant say “not significant”?
Small samples produce noisy rates. A +10% lift on 200 visitors per arm is well within chance; the confidence intervals shown will visibly overlap. The fix is more traffic per variant, decided before the test - not peeking until significance appears, which invalidates the p-value.
Can I compare more than two variants?
Yes - add more lines. Each variant is compared against the control (first line). Remember that testing many variants multiplies your chances of a fluke significant result; with 5+ arms, consider a stricter threshold like 0.01.