A/B Test Calculator

Analyze A/B test results instantly with a two-proportion z-test. Calculate conversion rates, lift, Z-scores, p-values, confidence intervals, and statistical significance — free and browser-based.

Variant A

Control

Visitors

Conversions

Variant B

Challenger

Visitors

Conversions

Confidence Level

Test Type

Decimal Precision

Calculation Steps

CRa = 420 ÷ 10000 = 4.20%

CRb = 510 ÷ 10200 = 5.00%

Pooled p = (420 + 510) ÷ (10000 + 10200) = 0.0460

SE = √(p(1−p)(1/na + 1/nb)) = 0.0029

Z = (CRb − CRa) ÷ SE = 2.7126

P-value = 0.0067

Result

Statistically Significant

Winner: Variant B

Conversion Rate A

4.20%

Conversion Rate B

5.00%

Relative Lift

+19.05%

Conversion Rate Comparison

Decision Summary

Variant B outperforms the other variant with a +19.05% relative change, and the difference is statistically significant at the 95% confidence level (p = 0.0067).

Statistical Details

Absolute Difference0.80%

Z-score2.7126

P-value0.0067

Confidence Interval0.22% to 1.38%

Test TypeTwo-tailed

Confidence Level95%

Export & Share

What Is an A/B Test Calculator?

An A/B test calculator is a free browser-based statistical tool that determines whether the difference in conversion rate between two variants — Variant A and Variant B — is statistically significant or just random noise. It answers the question every marketer and product team asks after running a split test: is this result real, or could it have happened by chance?

The calculator uses a two-proportion z-test, the standard statistical method for comparing conversion rates between two independent groups. Enter the visitor and conversion counts for each variant, and it instantly computes conversion rates, lift, a Z-score, a p-value, a confidence interval, and a clear significant/not-significant verdict — complete with a plain-language decision summary.

This tool is built for marketers, product managers, UX researchers, SaaS companies, advertisers, developers, eCommerce businesses, startups, and data analysts who need fast, reliable A/B testing insights without complex statistical software. It runs entirely in your browser — no data is ever sent to a server.

How the A/B Test Calculator Works

The calculator compares the conversion rates of Variant A and Variant B using a pooled standard error and a Z-score, then converts that Z-score into a p-value using the standard normal distribution. If the p-value falls below your significance threshold (1 minus your chosen confidence level), the result is declared statistically significant.

Core Formulas

Conversion Rate = Conversions ÷ Visitors

Pooled Rate (p) = (ConvA + ConvB) ÷ (VisitorsA + VisitorsB)

Standard Error = √(p × (1 − p) × (1/VisitorsA + 1/VisitorsB))

Z-score = (CRb − CRa) ÷ Standard Error

→Conversion Rate: The percentage of visitors who converted in each variant, calculated independently for A and B.
→Lift: The relative percentage change in conversion rate of B versus A — positive means B outperforms A.
→Z-score & P-value: The Z-score measures how many standard errors apart the two rates are; the p-value is the probability of seeing a difference this large purely by chance.
→Confidence Interval: A range around the observed difference in conversion rates, showing the plausible true difference at your chosen confidence level.

How to Use the A/B Test Calculator

Step-by-Step Guide

1Enter Variant A Data: Type the total visitors and conversions for your control variant (Variant A).
2Enter Variant B Data: Type the total visitors and conversions for your challenger variant (Variant B).
3Choose a Confidence Level: Select 90%, 95%, or 99% — 95% is the standard used by most A/B testing tools.
4Select a Test Type: Use Two-tailed to detect a difference in either direction, or One-tailed if you only care whether B beats A specifically.
5Read the Live Results: Conversion rates, lift, Z-score, p-value, and the significance verdict update instantly as you type.
6Review the Decision Summary: A plain-language summary tells you whether to trust the result or keep collecting data.
7Export or Share: Copy the report, download it as CSV, TXT, or JSON, print it, or share a URL with your inputs encoded.

Key Features

✓Two-proportion z-test with pooled standard error
✓90%, 95%, and 99% confidence levels
✓Two-tailed and one-tailed test support
✓Conversion rate, absolute difference, and relative lift
✓Z-score, p-value, and confidence interval of the difference
✓Color-coded significance badge and winner detection
✓Interactive conversion rate comparison bar chart
✓Swap Variants and Load Example one-click actions
✓Adjustable decimal precision (2–5 places)
✓Instant calculation with a 150ms debounce as you type
✓Shareable calculation URL using query parameters
✓Export report as CSV, TXT, or JSON, plus a printable layout
✓Calculation history — save and reload up to 20 past tests
✓Validation that blocks conversions exceeding total visitors
✓All processing runs locally — no data leaves your browser

Real-World Use Cases

Landing Page Redesign

A growth marketer tests a redesigned landing page (Variant B: 510 conversions from 10,200 visitors) against the original (Variant A: 420 conversions from 10,000 visitors). At 95% confidence, the calculator confirms the 5.00% rate is significantly higher than 4.20%, giving the team confidence to ship the redesign.

Inconclusive Email Subject Line Test

A lifecycle marketer compares two subject lines across roughly 3,500 recipients each, seeing open rates of 6.00% versus 6.20%. The calculator returns a p-value well above 0.05, showing the difference isn't statistically significant — the team decides to keep testing rather than declare a winner.

Checkout Button Color Test

An eCommerce team runs a large-scale test across 50,000 and 51,000 visitors on two checkout button colors, seeing 2,300 versus 2,520 completed purchases. The large sample size lets the calculator detect a smaller lift as statistically significant, informing a permanent design change.

Pricing Page CTA Test

A SaaS product manager tests two call-to-action wordings on a pricing page with a one-tailed test, since the team only cares whether the new wording increases signups, not whether it decreases them.

Ad Creative Comparison

A performance marketer compares click-to-conversion rates across two ad creatives using a 99% confidence level before committing additional ad spend, wanting a higher bar of certainty given the budget at stake.

Mobile App Onboarding Flow

A product team tests a shortened onboarding flow against the original, using the confidence interval of the difference to understand not just whether the new flow is better, but by roughly how much.

Tips & Common Mistakes

Pro Tips

💡Decide your sample size and test duration in advance using a sample size calculator — checking results repeatedly and stopping as soon as you see significance inflates your false positive rate.
💡Use a one-tailed test only when you truly don't care about detecting a decrease — most product and marketing tests should stay two-tailed.
💡A p-value just under your threshold with a small sample is fragile — favor tests with larger sample sizes for decisions with real business impact.
💡Look at the confidence interval, not just the significance verdict — a significant but tiny lift may not be worth the engineering cost to ship.
💡Run tests for at least one full business cycle (often one to two weeks) to avoid day-of-week or novelty effects skewing your conversion rates.

Common Mistakes to Avoid

✕Don't stop a test the moment it first shows significance — peeking at results repeatedly without correction dramatically increases false positives.
✕Don't declare a winner from a tiny sample size — a 60% versus 40% conversion rate means little with only 10 visitors per variant.
✕Don't ignore a wide confidence interval — it signals the true effect size is uncertain even if the p-value crosses your threshold.
✕Don't compare conversion rates measured over different time periods or traffic sources — that introduces confounding variables the z-test can't account for.
✕Don't confuse statistical significance with practical significance — a significant 0.1% lift may not be worth implementing.

Significance Reference Table

P-value	At 95% Confidence	Interpretation
p < 0.01	Significant	Very strong evidence of a real difference
p < 0.05	Significant	Standard threshold — reasonably strong evidence
0.05 ≤ p < 0.10	Not significant	Weak or borderline evidence — consider more data
p ≥ 0.10	Not significant	No meaningful evidence of a difference

Frequently Asked Questions

What is an A/B test calculator?

An A/B test calculator is a free browser-based tool that determines whether the difference in conversion rate between two variants is statistically significant. It uses a two-proportion z-test to compute conversion rates, lift, a Z-score, a p-value, and a confidence interval from your visitor and conversion counts.

How is statistical significance calculated in an A/B test?

The calculator computes a pooled conversion rate across both variants, derives a standard error from that pooled rate, and divides the observed difference in conversion rates by the standard error to get a Z-score. That Z-score is converted to a p-value, and if the p-value is below your significance threshold, the result is statistically significant.

What is a good p-value for an A/B test?

The most common threshold is p < 0.05, corresponding to 95% confidence, meaning there's less than a 5% chance the observed difference happened purely by chance. Some teams use a stricter p < 0.01 threshold for high-stakes decisions.

What is the difference between a one-tailed and two-tailed test?

A two-tailed test checks whether Variant B is either better or worse than Variant A, and is the standard, more conservative choice. A one-tailed test only checks whether B is better (or only worse) than A in one specific direction, which requires less evidence to reach significance for that direction.

What does 'lift' mean in A/B testing?

Lift is the relative percentage change in conversion rate between the two variants, calculated as (CRb − CRa) ÷ CRa × 100. A lift of +19% means Variant B's conversion rate is 19% higher than Variant A's, relatively speaking.

Why did my test show 'not statistically significant'?

This usually means either the true difference between variants is small or nonexistent, or your sample size is too small to detect the difference reliably. Use a sample size calculator before running your next test to determine how many visitors you need.

Can I use this calculator for more than website conversion rates?

Yes. The two-proportion z-test works for any comparison of two binary outcome rates between independent groups — email open rates, app install rates, sign-up rates, or click-through rates all qualify.

What does the confidence interval of the difference tell me?

It shows the range of values the true difference in conversion rates is likely to fall within, at your chosen confidence level. If the interval does not include zero, that supports a statistically significant difference between variants.

How large does my sample size need to be for reliable results?

It depends on your baseline conversion rate and the minimum lift you want to detect — smaller expected effects require larger samples. Use a dedicated sample size calculator before launching your test to plan an appropriate visitor count per variant.

Is my data private when using this calculator?

Yes. All calculations run entirely in your browser using JavaScript. Your visitor and conversion numbers are never transmitted to any server, stored in any database, or accessible to anyone other than you.

Who Uses This Calculator?

📈

Marketers & Growth Teams

Validate landing page, ad creative, and email subject line tests before scaling them.

🛒

eCommerce Businesses

Test checkout flows, product pages, and pricing changes with statistical confidence.

🖥️

Product Managers & UX Researchers

Confirm whether a new feature or flow actually improves conversion or engagement.

💻

Developers & Engineers

Verify experiment results before removing feature flags or shipping permanent changes.

📊

Data Analysts

Double-check experimentation platform results or analyze tests run outside a dedicated tool.