A/B test statistics

Pareto compares your A/B test variants side by side and recommends a winner once the result is reliable.

1. Open your A/B test statistics

There are two ways in. Both show the same numbers.

From the Statistics page

From the main menu, click Statistics > Pre-purchase.
Leave the campaign dropdown on All offers to see every A/B test in your store, or pick a single offer to see only its tests.
Scroll below the existing widgets to the A/B testings card. It lists your 5 most recent test runs.
Click View more to open the full table with every run.

$Pareto Statistics page with the A/B testings card below the standard widgets, showing a compact table of recent A/B test runs with period, status, views, conversion rate, orders, and winner columns\n$

The A/B testings card only appears if your store has at least one A/B test. If you have never run one, the Statistics page looks exactly as it does today.

From the A/B test archive

For a previous A/B test, from the main menu, click Offers.
Switch to the A/B test archive tab.
Find the test you want and click View statistics on that row.

This opens that run's details directly.

A/B test archive row showing the campaign name, A/B period, status badge, an "A/B winner A" badge, and a separate "View statistics" action icon

2. Read the A/B testings table

Each row is one test run — a single execution of an A/B test on a campaign.
You can A/B test the same campaign as many times as you like. Every run gets its own row, newest first, and running tests appear alongside ended ones.
Each row shows:

Campaign — the offer being tested.
A/B period — the run's start and end dates.
Status — Running or Ended.
Views — how many times each variant's widget was shown.
Conv. rate — orders divided by views.
Orders — orders attributed to the run.
Winner — the winning variant, once there is one.

The full table shows 25 runs per page.

3. Open a single test run

Click any row in the card or the full table to open that run's detail.
The detail has three parts:

Performance table — Views, Orders, Conv. rate, AOV, Revenue, RPV, and VS Other, split by variant A and variant B.
Statistical confidence bar — how confident Pareto is in the result, against a 95% threshold, with the leading variant marked.
Conversion funnel — Product viewed → Widget viewed → Engaged → Added to cart → Purchases, split per variant, so you can see where each variant loses customers.

A/B test run detail showing the Performance table with variants A and B compared across Views, Orders, Conv. rate, AOV, Revenue and RPV, a statistical confidence bar below it with a 95% threshold marker, and a per-variant conversion funnel at the bottom.

A/B numbers always cover the run's own A/B period. Changing the date range at the top of the Statistics page does not change them — it only re-scopes the general widgets. This is intentional: a test result only makes sense over the window the test actually ran.

4. Understand the recommended winner

Pareto picks the winner on revenue per view (RPV) — the average revenue each variant generates per person who saw it.
RPV is used instead of conversion rate because a variant can convert more often while making you less money. Revenue per view captures both how often people buy and how much they spend.
A variant is marked Recommended winner only when both of these are true:

Requirement	Why it exists
Each variant has at least 1,000 views	Small samples produce winners that reverse later.
Confidence is 95% or higher	Below this, the gap between variants is likely random noise.
Until both are met, Pareto shows the leading variant but does not call it a winner.

How it's calculated. RPV = revenue attributed to the variant ÷ that variant's views. Confidence comes from a Welch's two-sample t-test on per-view revenue, where confidence = 1 − p-value (two-sided). The threshold is 95%.

Pareto never applies a winner for you. Once you have decided, keep the variant you want in the A/B test overview or the campaign editor.

5. When no winner is shown

Not seeing a winner is a result, not a bug. There are three reasons.

What you see	What it means	What to do
Collecting data — keep the test running	At least one variant has fewer than 1,000 views. Confidence may be shown, but marked preliminary.	Let the test keep running. Do not act on the numbers yet.
No clear winner — both variants performed about the same. You can keep either one.	Both variants passed 1,000 views, but confidence is still under 95%.	Keep either variant. More data is unlikely to help — the variants genuinely perform the same. Test a bigger change next time.
Ended without a statistically reliable winner	The run ended while a variant was still below 1,000 views.	The result is inconclusive. Run a new test, and give it longer or more traffic.
A/B test run detail in the preliminary state, showing the confidence bar marked preliminary, no winner badge, and a "Collecting data — keep the test running" note below the Performance table.
If a variant has no data at all, its derived metrics show a dash (`-`) rather than zero, and no winner is declared.

6. What each metric means

Metric	Definition
Views	How many times the variant's widget was shown.
Orders	Completed, paid Shopify orders that applied that variant's discount. Reaching checkout without paying does not count.
Revenue	Total value of those attributed orders, in your store currency.
Units	Units sold across those attributed orders.
AOV	Revenue ÷ orders. Shows a dash if the variant has no orders.
Conv. rate	Orders ÷ views. Shown for reference — it is not what decides the winner.
CTR	Widget interactions ÷ views. Any interaction with the widget counts.
RPV	Revenue ÷ views. This is the metric that decides the winner.
VS Other	How this variant compares to the other one.

7. Good to know

Refunds and cancellations do not change the numbers. Figures are frozen when they are calculated, so a variant's revenue and order count stay as recorded even if an order is refunded later.
Tests always have exactly two variants. A and B only — Pareto does not support three or more variants in one test.
Dates use your store timezone. All A/B periods are shown in your store's timezone.
Revenue uses your store currency.
Statistics are available on every plan. Any store with an A/B test can read its results.
Deleted tests disappear from the list. If you open an old link to a test that no longer exists, you will see a note saying This A/B test is no longer available — the page still loads normally.
If you see a "Last updated" time, the numbers come from a cached snapshot taken at that moment rather than live data.

Updated on: 28/07/2026

Was this article helpful?

Thank you!