How do I measure whether email is truly incremental?
Use a holdout test: withhold a randomly chosen group from a campaign or flow and compare their purchases with the group that received it. The difference is the revenue email actually caused.
Attributed revenue counts orders that followed an email click or open, but some of those buyers would have purchased anyway. A holdout compares like with like and isolates the real lift.
Choose the holdout randomly and keep it large enough to see a difference, and run the test long enough to cover a normal buying cycle. Keep conditions identical apart from the email itself.
Even a rough test can reveal which automations are earning their place and which just claim credit for sales that were already going to happen.
In practice.
A store holds back a random slice of its audience from one promotional campaign. After the sales window closes, it compares purchase rates in the two groups. The gap is smaller than the attributed figure, so it reduces discounting on that campaign.
Want this handled for your store? See our klaviyo email marketing service or book a free audit.
Guides that go deeper.
Find out what your list is actually worth.
We review your flows, list health, authentication records and last ninety days of performance, then send back what we found.
30 minutes, no obligation, and the findings are yours to keep either way.