Before we get into the data: what is incrementality, and why does it matter?
Incrementality measures whether your advertising actually caused a conversion or whether that customer would have converted anyway, with or without seeing your ad.
Most campaign dashboards report attributed conversions: any purchase that occurred after someone was exposed to your ad, within a defined window. But attribution is not causation. A customer who was already ready to buy, who searched for your brand and then happened to see a retargeting ad before converting, gets credited to that ad, even though the ad did nothing. Your CPA looks great. Your incremental CPA is a different number entirely.
The only way to measure true incrementality is a holdout test: randomly withhold a statistically significant portion of your target audience from seeing the campaign, then compare conversion rates between the exposed group and the holdout group using your own transaction data. The lift between those two groups is your incremental conversion rate. The cost of producing that lift is your true cost-per-incremental-conversion — the number that actually tells you whether a channel is working.
This matters most when a channel gets cheaper. Falling CPMs make it tempting to scale spend. But if that spend isn’t driving incremental conversions, you’re scaling waste and doing it more efficiently than before.
CTV Is the Channel Where This Problem Is Playing Out in Real Time
Netflix’s new $19.99 ad-free standard plan is accelerating the migration of premium subscribers onto ad-supported tiers, and every other streamer is following the same playbook. The result: a surge of CTV inventory hitting the market just as upfronts lock in commitments, driving CPMs down and tempting performance teams to scale spend against a channel where cheaper impressions don’t guarantee cheaper incremental conversions.
Falling CPMs Don’t Mean Falling Cost-per-Incremental-Conversion
CTV CPMs have dropped meaningfully year-over-year as ad-supported inventory has outpaced demand — most platforms now offer CPMs under $30, down from $35–50 for premium inventory just two years ago. But CPM measures cost per impression served, not cost per conversion caused. When you factor in probabilistic household matching — IP-graph stitching, device fingerprinting — and platform-reported attribution that can’t isolate true lift, the gap between your dashboard CPA and your real incremental CPA may be widening as you scale into cheaper inventory pools with lower viewer engagement and weaker identity signals.
CTV Identity Resolution Is Probabilistic. Direct Mail Identity Resolution Is Deterministic.
CTV matches a household through IP addresses and device graphs that carry confidence intervals, not certainties. A single IP can map to a college dorm, a shared office building, or an outdated device cluster. Programmatic direct mail resolves identity against a USPS-validated physical postal address — the most stable household-level identifier in the U.S. — before a single dollar of production spend is committed. No probabilistic graph stitching, no cookie syncing, no device ID matching. The identity either passes deterministic validation or it doesn’t enter the mail stream.
That distinction is the difference between measuring incrementality against a clean control group and measuring it against a noisy approximation.
Why Holdout-Based Incrementality Tests Require Deterministic IDs
A valid incrementality test needs a clean split: an exposed group and an identical holdout group that you can confirm never saw the treatment. CTV can’t guarantee holdout purity because household-to-device mapping is inferred, not verified. Exposed and holdout groups bleed into each other at rates that inflate measured lift. You may think you’re measuring a clean experiment. You’re not.
Direct mail holdouts are airtight. If a postal address is in the holdout cell, that household did not receive a mailpiece. Period. The brand controls both sides of the measurement equation, and because Postie’s matchback attribution uses the same deterministic address at send and at conversion, there’s no inference layer where signal can degrade between measurement points.
This is why the most rigorous CTV incrementality designs use a deterministic channel-like programmatic direct mail as the measurement baseline. It provides a ground-truth conversion rate against which to benchmark CTV’s claimed lift.
The Q3 Planning Decision Framework
Before increasing CTV allocation on the basis of lower CPMs, run a holdout-based incrementality test that isolates CTV’s true lift from organic demand and cross-channel contamination. Use programmatic direct mail as the baseline to validate whether CTV’s platform-reported conversions hold up against a channel where you control identity, holdout integrity, and attribution end-to-end.
How to structure it:
- Randomly withhold a statistically significant subset of your mail audience as a holdout
- Run the direct mail campaign to the exposed group concurrently with your CTV spend
- After the campaign window, compare conversion rates between the mailed group and the holdout group using your own transaction data
- Compare incremental CPA across both channels using the same methodology
The output isn’t just a verdict on CTV — it’s a measurement infrastructure that holds every channel in your mix accountable to the same standard. Marketers who establish that baseline before upfront commitments lock will allocate from evidence. Everyone else will allocate from dashboards.
Upfront commitments don’t wait. The baseline needs to be built before the budget is locked — not after.
Postie gives performance teams the deterministic targeting, airtight holdout design, and matchback attribution infrastructure to build incrementality baselines that hold every other channel in your mix accountable. [See how Postie’s incrementality testing works →]
