Lesson 23 of 30 · 11 min
Incrementality: what advertising actually caused
The only measurement that answers the question everyone is really asking.
The question attribution cannot answer
Attribution asks which touchpoint deserves credit for conversions that happened. Incrementality asks a harder question: how many of those conversions would have happened anyway? A campaign can be credited with thousands of sales and cause almost none of them.
Holdouts are the cleanest evidence
Withhold advertising from a randomly chosen slice of the audience and compare outcomes against those who saw it. The difference is caused by the advertising, because the only systematic difference between the groups is exposure.
Geo tests when user-level holdouts are impossible
Where you cannot split an audience cleanly, split the map instead: run in some markets, stay dark in comparable ones, and compare. It is coarser than a user-level holdout and far more practical for CTV, audio and out-of-home.
Retargeting is where the surprise usually lands
Retargeting reaches people already close to buying, so it is credited generously and is often the least incremental line in the plan. Testing it is uncomfortable precisely because it so often shows that much of that revenue was arriving regardless.
Test rarely, but properly
Lift tests cost real money in withheld reach and need enough volume to detect a difference. Run them on the big questions — a major channel, a large budget line — rather than continuously on small ones where noise will swamp the result.
Takeaway
Incrementality measures what your advertising caused; attribution only measures what it was near.
Check yourself
No score, no signup — pick an answer to see why it's right.
Question 1
What does a holdout test measure that attribution cannot?
Question 2
Why is retargeting often the least incremental line in a plan?