One of the brands we work with grew online revenue by 60% in a year. That is a great result but it doesn’t actually tell us much about why or how to drive continued growth.

“We grew 60%” is a result but it misses the explanation.

And without the explanation, growth is just something that happened to you. It becomes impossible to reliably repeat, impossible to scale with confidence, and dangerous to build next year’s plan on.

So we dig into the reasons why, what were the levers that really moved the dial to drive this growth.

We can compare two marketing mix model runs, one from the start of the period and one from the end, then this allows us to break down the full revenue uplift into its individual drivers.

A third of the growth would have happened anyway. We can’t really claim the credit for it, but we definitely need to understand it.

Around a third of the total uplift (28% from underlying base growth, plus another 5% from smaller external factors that happened to the brand rather than things anyone decided. Growing organic interest, seasonality, market movement, competitor behaviour.)

Even if we had not made any marketing changes, we would have had 33% of the YoY growth. It means we can’t really claim credit for it, but it is really important that we understand it. We need to know the drivers of this organic growth so we can know whether to forecast it for next year.

The remaining two thirds came from marketing decisions. And they fall broadly into three categories.

The biggest single driver, 43% of the growth, was spend volume.

We spent more on marketing spend and that generated more revenue. It helps prove that the additional spend drove incremental growth, and it helps prove that this additional spend continued to perform strongly even at higher diminishing returns levels.

It gives us a clear lever we can continue to pull and understand the expected impact on the business at each spend level.

The other two markets are arguably even more important.

7% of growth from improved media efficiency. The money we spent worked harder than it did last year. This is a direct impact of how the campaigns are activated.

We can isolate this by comparing the coefficients of the two models rather than simply the revenue contributions. At every level of spend, our marketing investment is working harder and generating stronger returns.

17% of growth comes from optimisations. Primarily spend optimisations to focus investment on channels which have stronger incremental revenue impact. Moving investment out of lower returning channels and into stronger incremental return channels.

This is different to the media efficiency point, the channels themselves are working 7% harder but we get an additional 17% of growth from knowing where to put our money to make it work hardest.

Combine the two and we get 24% of YoY growth simply by making the marketing spend work better, it doesn’t require any external factors and it doesn’t require any additional spending. These two effects compound the external factors and increased spend to outpace the market.

It is really important to understand what is driving the growth, because that 60% YoY number on its own can be misleading.

Three brands could all see 60% YoY growth but from very different reasons:

1. The first rode a category tailwind.

2. The second bought growth purely by increased investment

3. The third drove growth through better decisions

All three see significant YoY growth, but they are all faced with a different outlook for the future.

The first brand needs to rely on an ever growing category.

The second needs to continue to spend more.

The third needs to continue to find more efficiency.

These are very different challenges and very different game plans. Ideally (as we see in the example above) we get growth from a combination of the three, which allows us to continue to outpace the market and outmaneuver the competition.

This is why “did we grow?” is not even half the question.

The real question is “how much of our growth did we cause?” and “how much of our growth can we continue to replicate?”

Outside of MMM, we can see some clear indicators that help show that the growth is driven by things we control rather than market factors.

Share of search is a great measure for this.

We can see the brand search demand increasing significantly year on year in every month of 2025 vs 2024 equivalents. The growth YoY increases in Q3 and Q4 as the growth begins to compound.

Conversely we see that category growth is much less consistent, much smaller and shrinks to flat growth in Q4.

So clearly the underlying growth isn’t purely being driven by category growth, and must be coming from some controllable factors.

Once you know which drivers caused your growth, planning stops being guesswork.

You can forecast more reliably, because you know how much of last year’s growth was controllable and what the relative contribution was from each controllable factor.

You also know how susceptible you are to external factors, so if category growth falls above or below expectations then how much will that impact our revenue growth.

And you can have a much clearer answer to “what will happen if we invest more / less”. We can build scenario plans based on understanding both the controllable and uncontrollable impacts, with forecasts built from bottom up rather than top down.

The takeaway

Topline growth is a result not an analysis.

Understanding the impact of both the controllable and uncontrollable drivers is the data which allows you to outpace the market and the competition.

Next time you see a year on year growth number, ask three questions of it. How much of this would have happened anyway? How much did we buy? And how much did we earn through decisions we could make again?

FAQs

What is growth decomposition in marketing?

Growth decomposition is the process of breaking down the total revenue growth into its individual drivers. Typically these include base growth, spend volume, media efficiency, and specific marketing decisions or optimisations. You can do this using marketing mix modelling (MMM). Instead of just reporting the year on year growth rate, we can quantify how much growth came from market conditions, how much was bought with extra budget, and how much was earned through better decision making.

How do you know if marketing actually caused your revenue growth?

We do this through Marketing Mix Modelling (MMM). Compare two marketing mix (MMM) model runs from the start and end of the period and compare both the contributions from each variable and the change in coefficients for each variable. In one retail example, revenue growth split into roughly a third that would have happened anyway (base growth and external factors), 43% bought through increased spend, and 20% gained through media efficiency and optimisations.

What is base growth in marketing mix modelling?

Organic base is the revenue a brand would generate without any marketing activity. It is modelled from organic brand demand, seasonality, category movement and market factors. In growth decomposition, base growth separates “things that happened to the brand” from “things the marketing team did”. It is crucial to understand what is the cause of our decisions, and what is the result of things which happened to us. Sometimes we can do everything right and still get a bad outcome (due to external factors), sometimes we can do a lot of things wrong and still come out positive (for the same but opposite reason!)

Nothing

A little

A lot.

Actually usually all three of those things happen. And usually in that order.

And the order is dangerous. Because the first signal is “Nothing changed”, so the marketing spend wasn’t doing anything.

It is a problem baked in by reports which run weekly over outcomes which happen monthly, quarterly or yearly.

The damage doesn’t happen on the day you cut spend, the next day’s results will often look fine.

The damage comes in installments, and those installments are baked in from the point of the budget cut even though some of them won’t be seen for several months.

The short term redistribution of revenue vs the long term decline in revenue

When budget is first reduced, the attributed revenue to that channel drops immediately (as you would expect), but total revenue is often relatively unchanged, especially if those cuts come from upper funnel channels (which is where they are usually taken from first.)

We can see it clearly in an example in the chart below. This is anonymised but real data of a brand who has made these cuts and seen these impacts over the past 2 years.

In months 1-8 Marketing spend has been consistently around £150k /month. Total revenue is on a slow increase but staying around £800k-£1m.

Marketing spend goes through two cuts, once in Month 9, reducing down to £125k/month, then in month 16 reducing down to £75k/month (half the original monthly spend.)

Let’s focus on the total revenue lines in Months 9 and 10 and Months 16 and 17. Marketing spend has reduced significantly, but total revenue was flat (it actually increased in month 10, mainly due to some market fluctuations.)

Marketing spend cut and total revenue didn’t drop. Seemingly the marketing wasn’t working.

The insight from Month 9 gave the confidence to further cut spend in Month 16. The revenue performance in Month 17 gave the confidence that the decision was correct.

Until it wasn’t correct.

By Month 21 it was clear that the false confidence from short term metrics was driving some bad consequences.

If we split the Revenue line into Organic and Marketing Revenue then the picture gets a bit clearer.

When marketing spend is cut the Marketing Revenue drops straight away, we see it in both Month 9 and Month 16.

But some of that drop is longer term demand which still converts through organic channels, we can see Organic Revenue actually increase in Months 9&10 and in Month 16. So this softens the overall impact on Total Revenue.

But when we look at both the Organic Revenue and Marketing Revenue lines on a longer timescale the impacts are much larger.

From Month 11, Organic Revenue drops by 40%. The initial lagged demand has run out, and with reduced marketing investment there is less new demand filling the gap. The new level of marketing spend continues to look efficient, but isn’t building long term growth.

From Month 18, Organic Revenue drops a further 25%. This time Marketing Revenue also declines with it, without the built up lagged demand the Marketing Revenue has less to fall back on.

We can see those three time periods of impact on the 50% budget reductions:

Nothing happens: Months 9 & 10 and 16-17 see little impact on Total Revenue. Particularly for the first cut which was more focused on upper funnel channels. The long term lagged demand compensates for the reduced Marketing Spend and gives an appearance that nothing has changed.

A Little happens: Months 11-15 see a decline in Organic Revenue, but Marketing Revenue on the remaining spend (focused more on bottom of funnel channels) stays strong which softens the impact on total revenue.

A Lot happens: Months 20-24, Organic Revenue continues to fall at around 10% per month, along with the sustained 40% drop in Marketing Revenue. Compounds to a 42% drop in Total Revenue from Month 15 to Month 24

The timeline really matters

Initially things look fine when the marketing spend is decreased. Marketing Revenue decreases but it is initially covered by Organic Revenue.

Total Revenue is flat for the first two months after the spend decreased. This highlights the challenge with running uplift tests on high consideration products, most brands will not commit to a holdout test beyond two months, yet for this business we can see that the negative impact of the reduced spending is only present on this longer timeframe.

A holdout test across a 6 week period would show that the marketing spend had no impact, and creates the false confidence to reduce investment. To reliably see this impact on a holdout test we would need to wait until Months 4 or 5, I don’t know many brands who are prepared to take that level of opportunity cost.

But that doesn’t mean we need to wait several months to see the negative impact. The organic revenue line drops after month 2, so we get an early warning of what is to come.

FAQs

What happens when you cut marketing spend in half?

Three things, usually in this order: nothing, a little, then a lot. Total revenue typically holds for around two months as lagged demand converts, organic revenue then declines, and the full impact lands months later with both organic revenue and paid marketing revenue declining.

Why does revenue stay flat after a marketing budget cut?

Because built-up demand from previous marketing keeps converting, often through organic channels. Attributed channel revenue drops immediately, but organic revenue temporarily rises to soften the total. Which often creates a false signal that the marketing wasn’t working.

How long does it take to see the impact of cutting marketing spend?

The first reliable signal is organic revenue, which in our example began falling two months after the cut. The full revenue impact took 5+ months to become undeniable, long after weekly reports had thought there was minimal impact.

Do holdout tests show the true impact of marketing cuts?

Not on high-consideration purchases. While a six-week holdout may show marketing spend had no impact, the damage may only becomes visible from months 4–5, and few brands will carry that opportunity cost in a test.

What is the early warning sign that a budget cut is hurting the business?

Organic revenue. It starts declining around two months after a cut, well before total revenue confirms the damage. Watch organic and direct demand, not blended efficiency metrics, in the months after any budget change.

Subscribe now

There are a number of SaaS tools which promise to give it to you.

A single source of truth takes away all the questions around what is and isn’t working.

Makes marketing measurement and optimisation easy.

Except it doesn’t work.

Because marketing measurement is tough, and real people are irrational and confusing.

There is a lot of noise in any dataset and it is virtually impossible to create truly clean uplift experiments.

Any single source of truth is wrong for at least one of the things that are important.

A few weeks ago I wrote about the three speeds involved in making marketing decisions (available here: https://danwilson531789.substack.com/p/making-decisions-at-3-speeds), and why using one method across all three breaks down.

The same problem shows up again once you change the question from “how fast” to “what kind.”

This month’s revenue, next quarter’s leads and long term brand health are not the same thing. They don’t work on the same timescale, they don’t respond to the same marketing levers.

Forcing one metric, measurement or approach to answer all three questions produces a bad answer to at least one or two of them.

Why one measurement model can’t do three jobs

We can isolate these questions and answer each of them independently.

Multiple sources of truth.

Doesn’t sound as catchy, requires a bit more nuance and judgement in tradeoffs and flexibility (how much do I favour one measure over another when they compete or disagree with each other.)

But they do give a more accurate answer to each challenge.

If we stick with the three core outcomes (this is taken from a live example for a brand we help with this challenge):

• In month revenue,

• Leads we expect to convert this quarter,

• Long term brand growth

All three are linked to driving revenue outcomes, but at different speeds and through different methods.

Each of our measurement techniques could apply to all three of these outcomes, but equally each have weaknesses across each measure.

Attribution will give a fast and tangible measure for revenue and leads, in platform brand uplift studies can give a directional steer towards long term brand growth. But all are attributed on a short time frame and skew towards correlation rather than causality.

Uplift tests can give a more causal view of each metric but they are limited in timeframe (because no one tends to run holdout tests over multiple months even when purchase decisions can frequently cover several months). And these tests are simply not possible for a number of really important channels, we can’t control who sees our PR or organic influencer activity.

Timeseries models, such as MMM, give a directional causal answer while accounting for some of the things we can’t realistically keep constant, and can do so without the opportunity cost of uplift tests. But require some significant interpretation and judgement of when to trust the direction and when to validate elsewhere.

But whichever method you choose, will have weaknesses. There should never be a single source of truth.

When the real marketing truth is unknowable we need to build up a body of evidence to get closer to an answer and reduce the risk in making big decisions.

Think of it like building a case in court, you want multiple forms of evidence using different base facts to support the same theory. The more sources you have, and the more variety of data evidence you can build then the more confident you should feel about making a decision.

We can work through those three main goals, along with what we learn from each measurement method, with the sample findings we have seen implementing these in the real world:

If we want to understand the revenue incrementality of our paid search campaigns then geo uplift tests and MMM work really well together:

• The uplift tests show the revenue that happened with and without the paid search activity in two similar but distinct geo regions

  • Uplift tests show that our non brand search campaigns are 70% incremental at average spend level during non-peak periods

• The MMM should show a similar modelled uplift but will allow you to scenario plan this out to different spend levels across different market conditions and different spend levels (adding back some of the changing context that we tried to remove from the uplift test.)

  • MMM allows us to extrapolate that 70% incremental CPA to expected performance when we double spend, when competitors are more aggressive or when we go into peak seasonality.

If we want to understand the biggest drivers of in quarter converting leads, then attribution and uplift tests help cover each others weaknesses:

• The attribution gives the granularity of which individual source and tactic drove each individual lead, and allows us to match these to future revenue to isolate LTV back to attributed source

  • Attribution tells us that 50% of our converting leads come from Google Pmax campaigns

• Uplift tests allow us to understand the underlying incrementality of each lead source, a series of structured tests will allow us to isolate incrementality down to tactic and channel level

  • Uplift tests tells us that 30% of these Pmax leads would have come to us anyway, so we know that the net impact on converting leads from Pmax is 35% of leads (70% of 50%)

When we want to understand the drivers of, and impact of, increased awareness then we want to pin together brand studies and MMM:

• The brand studies give us a clear picture of how users perception towards the brand is changing over time, survey responses can tell us which activations they recall strongest

  • Brand tracking tells us that our unprompted awareness grew by 18% after an OOH and CTV campaign

• MMM can allow us to match these increased awareness measures back to incremental revenue across multiple timeframes

  • MMM allows us to understand that the 18% increase in unprompted awareness made our search campaigns 26% more effective, increased organic revenue by 12% in 3 months and 36% over 12 months.

These multiple sources of truth don’t make the picture less clear. They do the opposite.

The multiple sources of truth tell us when an impact can be trusted, and when we need to look harder.

When we build a body of evidence across multiple sources and multiple methods, we reduce the impact of the weaknesses of each method.

We can still make fast decisions from the real time data sources, but they are overlaid with the contextual and causal measures from other methods to align those fast decisions with long term incremental business impacts.

A single source of truth gives you an easy answer. A body of evidence gives you the right one.

FAQs

Should marketing measurement have a single source of truth?

No. Every measurement method (attribution, uplift tests, MMM) has structural weaknesses, so any single source of truth is wrong about at least one question that matters. Reliable measurement comes from multiple methods whose weaknesses don’t overlap.

Why doesn’t a single source of truth work in marketing?

Because in-month revenue, next quarter’s leads and long-term brand health operate on different timescales and respond to different marketing levers. One metric forced to answer all three produces a bad answer to at least one of them.

What are the main marketing measurement methods and their weaknesses?

Attribution is fast and granular but skews to correlation over causation. Uplift (holdout) tests are causal but time-limited and impossible for channels you can’t control, like PR or organic influencer activity. MMM is directional and causal but needs significant interpretation and judgement.

How do geo uplift tests and MMM work together?

The geo test measures revenue with and without a channel in two comparable regions and gives a causal read. The MMM should show a similar uplift, then extends it: scenario-planning the result across different spend levels and market conditions the test had to hold constant.

How do you measure the revenue impact of brand awareness?

Pair brand tracking with MMM. Brand studies show how perception shifts over time and which activations people recall; MMM links those awareness movements back to incremental revenue across multiple timeframes.

Subscribe now

I’ve seen both sides of this story multiple times across multiple businesses.

Efficiency metrics improve, month after month. CAC falls. ROAS climbs. The board deck looks great.

Then, months later, growth runs out and nobody can explain why.

The missing factor is organic demand. How many people would have bought from you without the short term performance marketing.

The trap of paid efficiency

I spent a few days with a business last week. Blended CAC down 57%. Blended ROAS up 126%. On paper, the best the brand has had.

Now look underneath it. Upper funnel marketing had been paused six months ago (they used to run a mix of OOH, TV and CTV.) Brand search fell 25% in response and has never recovered. Share of category search has sat at around 6%, when it used to be around 10%.

Put those two facts on the same chart and the “best year ever” reads differently. Paid efficiency improved because paid media was harvesting a shrinking pool of demand more effectively, not because it was creating more of it.

We can see that play out on the chart. All three colours show indexed values (Jan 2023 is 100 for each metric, every other month is an index against that value.)

We can see steady performance through 2023 and start of 2024 on all three values.

From Dec 2024, we can see the green line (Organic Brand Demand) and the orange line (Marketing ROAS) start to follow different paths. The green line consistently drops, initially down to 80’s then down to 50’s (half of the Jan 2023 volume), meanwhile the orange line starts to increase, up to 130 then shooting up beyond 250 by the middle of 2025.

To first view, it is a great story. Paid marketing driving ever efficient returns.

But the purple bars (performance marketing revenue generated) show the real impact of that declining green line. Initially it drops, and the purple bars stay high. Spending less to achieve the same outcome. But six months later the drop in the green brand demand starts to bite. Suddenly the orange line isn’t growing because the marketing is more efficient, it is growing because the demand isn’t there to drive more revenue (and spend more money at acceptable efficiency.)

While digital marketing efficiency looked to be improving the underlying brand demand which actually drove the business quarter over quarter had shrunk.

It is also true in the other direction

A different brand shows the opposite side of the coin.

Organic Demand has shown steady and consistent growth, with the green line reaching twice the Jan 2023 level.

The orange ROAS line peaked in the middle of 2024, before showing a steady performance through 2025 and 2026.

But the revenue contributed by marketing campaigns (the purple bars) continues to grow and grow, following (with a lag) the growth in the organic demand (green) line. By 2026, the revenue generated by marketing had reached more than 4x the Jan 2023 level.

Rather than creating increasing dependence on a decreasing pool of prospects, the organic demand is allowing the paid marketing to continue to drive growth at strong efficiency.

The same story across multiple brands and various categories

We’re seeing this pattern show up across a variety of brands and businesses.

Brands who have been coached into following digital attribution signals. Short term measures of efficiency rather than longer term measures of marketing effectiveness.

Marketing ROAS improves, so it looks like the marketing is growing. But without the underlying brand growth, the marketing ROAS does not necessarily align to long term brand growth.

In our research, brand-tracking data which shows a 10% increase in Awareness is associated with a 20% uplift in organic baseline revenue in subsequent months.

This increase in Awareness is driving the underlying organic growth which aligns to long term success.

Source: Charlie Oscar & Tracksuit Study

It is easy to test and prove yourself

Top level correlation analysis on overall business performance along with these tracking metrics will easily validate the numbers which actually matter.

It doesn’t need any sophisticated modelling or fancy SAAS subscriptions, just align the numbers, add some lags and cumulative fields and find what actually drives long term business success.

A simple step by step below:

• Take either weekly or monthly aggregate data on orders/revenue. Weekly if you are in a fast moving category with short decision cycles, or monthly if you are at higher price point, longer consideration purchases.

  • I wouldn’t recommend daily level analysis, there is too much noise in the data and it will tend to be too erratic

  • You will need a minimum of 12-18 months of data to get any meaningful correlation. Ideally on monthly data you need 3-4 years.

• Build weekly or monthly inputs for organic metrics and paid metrics to match your revenue data

  • Organic metrics: should include organic search sessions, direct sessions, email signups. And any specific engagements which are core to your business and purchase cycle.

  • Paid metrics: Paid marketing driven revenue, ROAS, paid marketing sessions.

• Build some immediate and lagged correlation metrics

  • In month / in week, how does January engagement correlate to January total revenue

  • Lagged metrics, how does January engagement correlate to February revenue

  • Cumulative metrics, how does January engagement correlate to the sum of January, February and March revenue.

These lagged and cumulative correlations will help isolate the metrics which actually drive longer term brand growth.

Crucially the organic metrics are reactive to paid marketing. When you invest in demand generating marketing, some of that impact will show up in increased organic traffic.

If the stronger correlation is on the organic metrics then it doesn’t mean your marketing isn’t working.

It usually means exactly the opposite, your marketing is driving the organic uplifts which grow both short term and long term revenue.

If you know what really matters for long term growth, then you can remove reliance from the short term immediate spikes and build sustainable long term brand growth.

FAQs

Is ROAS a reliable measure of business growth?

No. ROAS measures how efficiently paid media converts existing demand. It says nothing about whether demand is growing. ROAS can rise while the business shrinks, because paid media harvests a shrinking pool of demand more efficiently.

What is organic brand demand?

Organic brand demand is the volume of people seeking your brand without needing paid marketing. Organic demand usually shows up in brand search, direct traffic, organic sessions, email signups. It’s the pool paid marketing builds apon, and the strongest leading indicator of long-term revenue.

Why did our ROAS improve while revenue declined?

Usually because upper-funnel investment was cut. Efficiency metrics improve immediately as spend concentrates on the highest-intent buyers, but the demand pool stops refilling. In one case we analysed, blended CAC fell 57% and ROAS rose 126% while brand search fell 25% and share of category search dropped from ~10% to ~6%.

How does brand awareness affect revenue?

In the Charlie Oscar × Tracksuit study of 8 brands, every 1-point increase in awareness was followed by a ~2% uplift in organic baseline revenue two months later, consistent across 4 categories.

How can I test whether organic demand or paid efficiency drives my growth?

Aggregate 12–18 months of weekly (or 3–4 years of monthly) revenue data, build matching organic and paid metric series, and run in-period, lagged and cumulative correlations. No modelling platform required.

Does strong organic correlation mean paid marketing isn’t working?

No. Usually the opposite. Demand-generating paid marketing shows up as organic uplift. If organic metrics correlate hardest with revenue, your marketing is likely creating the demand that drives both short- and long-term growth.

Subscribe now

Performance media can point to clicks, conversions and ROAS. (While I don’t agree with making decisions on this data, it is widely understood and accepted as a measure of value.)

Brand investment rarely has an equivalent commercial story, so when the budget conversation gets tough, brand loses first.

We have just finished a cross-brand study with Tracksuit, joining brand health data to search behaviour to MMM outputs, across beauty, wellness, banking, travel, food & drink and athleisure.

We wanted one answer: can brand investment be proven with the same rigour we apply to performance marketing? Yes.

And the pattern connecting what people think, what they do, and what lands in revenue was more consistent than I expected.

Can we link what people believe to how they behave?

Most businesses fall into one of two camps. They invest in brand on instinct, or they wait for proof before investing at all. Neither gives a marketing leader the confidence to defend a budget line in a finance meeting.

The answer usually lies in joining three datasets that are almost always kept apart: brand health, search behaviour and revenue modelling.

Of all the brand metrics we tested, investigation, people saying they’d actively research or compare your brand, was the strongest near-term predictor of what happens next.

A 10% rise in investigation was associated with a 50% increase in branded search volume, arriving on a predictable lag: investigation moves in month one, branded search follows in month two, and keeps accelerating into month three.

That lag is the whole point. Most teams only look at search performance once it already shows up in the marketing reporting. By then you have already missed the factors which are actually driving the brand demand. Investigation gives you weeks of warning before it shows up anywhere near a revenue dashboard, which makes it a really valuable measure of future performance.

Preference, the shift from “I know this brand” to “I’d choose this brand”, had a huge commercial impact, but doesn’t show up in any performance dashboard. A 10% rise in preference was associated with a 27% increase in share of search, and the effect held across every category we tested, from fashion to pet care to finance.

What drives preference is the more interesting finding. Across categories, it wasn’t functional product claims doing the work. It was trust, affinity, passion and a feeling of “premiumness”. In the athleisure data specifically, “is a brand I trust” and “is worth paying more for” outscored practical attributes like sizing or clothing quality.

It is a really tough thing to measure and value, but users can tell you that is driving their preference, and we can measure the value of preference. Emotional connection is harder to build, but drives long term sustainable commercial impact.

Not all awareness is equal. A 10% rise in awareness was associated with a 20% uplift in organic baseline revenue, the demand you generate without paid media touching it at all.

Organic revenue is crucial for any brand that wants to become really big. Long term sustainable growth relies on building an organic base which can compound the paid marketing performance.

But Awareness depth matters more than breadth. Someone who only knows your name gives you an in month bump in search volume, but minimal organic revenue impact . Someone who knows you well gives you a delayed but much stronger lift in share of search and organic revenue, arriving two months later.

So the headline: brand metrics are leading indicators. Revenue is the lagging one. Treated that way, brand tracking stops being a retrospective report card and becomes an early warning system for where demand is heading.

The brands that win don’t tend to do drastically different things to other brands. But they do tend to have a clearer and faster view on what is and isn’t working (and a clearer understanding of how to measure what is working.)

Both brand tracking and incremental revenue measurement should be treated as always on measures. Not something to run once or twice per year.

The leading indicators are only valuable if you see them fast enough to do something about it, otherwise it is an interesting but backwards looking datapoint.

Track brand health continuously, not once or twice a year, because you can’t act on a signal you can’t see. Build full-funnel measurement that separates organic baseline revenue from paid contribution, because single-channel attribution misses the story entirely.

Match the brand measures and the revenue impacts together and you should have a dynamic view of what is moving today as well as what is moving the next few weeks.

It will let you make todays decisions based on tomorrows expected performance.

Subscribe now

The channel you’re running on is just the pipe. The creative is the water. Measuring the pipe without measuring what’s flowing through it tells you almost nothing about why performance is up or down.

Why is this campaign working? Or not working?

That question is impossible to answer if you can’t connect creative to media.

The inconvenient truth from CO’s data: creative quality is one of the strongest predictors of incremental revenue, and it’s almost entirely absent from how most brands report on campaign performance.

The metric that most use to judge creative is the wrong one

The most commonly used creative performance metric is CTR. It’s in every platform. It’s easy to compare across campaigns. It gives you a number for every piece of creative, which feels like measurement.

The problem: CTR shows a weak correlation with incremental revenue.

The second most common metric, click attributed conversions or revenue. It is a reflection of exactly the same problem.

Charlie Oscar analysis on Meta campaigns shows that when we compare CTR rankings of creative assets against their modelled incremental revenue contribution, the correlation is low.

A high-CTR creative simply isn’t reliably a high-revenue creative.

The metrics that actually predict incremental revenue are view rate, reaction rate, and save rate. These measure whether someone genuinely engaged with the content not whether they reflexively clicked through.

A piece of creative with a high CTR might be a great direct-response asset. It might also be a misleading thumbnail that drives clicks with no downstream intent.

CTR cannot tell the difference.

80% of paid social impact is passive, for people who never click on ads. CTR and click derived conversion metrics will never see that value. It means that every creative optimisation choice is made on a misleading 20% of impact.

Creator campaigns are being systematically undermeasured

Most brands measure creator and influencer campaigns through tracked links and promo codes.

These capture roughly 20% of the real commercial impact.

The other 80% is indirect. It works through brand perception, social proof, the downstream effect on how audiences respond to paid campaigns, and the organic reach effect on branded search and direct traffic.

Charlie Oscar runs a lot of MMM studies on creator campaigns, both organic reach and paid amplification. We see this consistently across every creator MMM study we run, tracked links can only see demand that clicks through to a conversion.

They can’t see demand that was created by a piece of content and then converted through a completely different channel three weeks later.

Strongest content makes your paid media work harder

Strong content, particularly creator content, drives synergy effects on paid media amplification.

It makes every pound and dollar of marketing spend work harder.

When creator content is run as paid ads, Meta performance improves 30%, TikTok by 40% and YouTube by 25%.

This is the connection between media and creative reporting. The media performance improved, but not by changing tactics or adjusting settings but by changing the quality of engagement and the data signal being fed to the platform algorithms.

You have to be able to connect these reporting outputs across media and creative to be able to identify when this performance is driven by changes in media tactics and when they are driven by changes in creative effectiveness.

As these measures come from MMM (COmpass), they are aligned to incremental business impact, not platform attributed response.

The false choice between data and creativity

The data versus creativity debate is one of the more persistent false dichotomies in marketing.

Data doesn’t constrain creativity. It tells you where creative risk is worth taking. Playing it safe (running only proven formats, well-worn messaging, low-risk executions) doesn’t produce stability.

It produces slow, invisible decline. The in-platform metrics look fine. The brand slowly becomes less interesting to more people.

If you can isolate the real impact of both creative and media, then you know which of your changes has moved the dial. Moving the dial on the real business numbers that matter

Run your own correlation studies, which creative metrics actually align to your business performance (Business performance, not platform reported performance). It won’t be CTR. It will likely vary by platform.

Use those metrics to isolate when your media performance is improving due to media choices and when it is improving due to creative effectiveness.

Monitor brand health as a downstream signal of whether the creative is working above the purchase funnel.

And when the data tells you an unexpected or risky creative format is performing, let it perform. Advertising used to be risky, people used data as an excuse to be safe. But the most effective performance continues to come from managed risk.

FAQ

How do I measure the indirect impact of creative and creator campaigns?

Charlie Oscar runs MMM to show both the direct and indirect impact of creator campaigns, which is the most effective way to understand real business impact of creators. You can validate these outputs through brand tracking and uplift studies (for paid amplification) .

If CTR is a bad creative metric, what should I use instead?

This will vary by brand and platform. But there are several better alternatives. For video: view rate (% who watched past 3 seconds), completion rate, and save rate. For static: reaction rate (likes/shares divided by impressions), save rate, and comment quality (LLMs are good at classifying these comments).

How do I separate channel performance from creative performance in my reporting?

The simplest method: hold the creative constant and vary the channel, or hold the channel constant and vary the creative. Platform A/B creative tests do this within a channel. Cross-channel holdouts do this within a creative. These tests are messy, but give good directional signal. MMM is a great method for increasing the speed and scale of this measurement without increasing the opportunity cost through A/B tests.

Is brand health tracking worth the cost for a mid-size brand?

Yes, Charlie Oscar and Tracksuit produced research to prove why. A 1% movement in consideration generates a measurable 3.5% uplift in branded search in the same month. A 1% shift in preference shows a 2.7% share of search gain. These are leading indicators that platform attribution will never surface. If you want to drive long term sustainable growth then you need to generate brand uplift, not just platform performance.

Subscribe now

Is this an awareness campaign, or a conversion campaign?

That question is a problem. It creates bad incentives.

Split influencer into awareness and conversion and you tend to get campaigns that do neither particularly well.

Awareness gets measured on reach and engagement, with almost no accountability for what it generates commercially. Conversion gets measured on click or code based orders, which misses most of the value influencer actually creates.

Both end up measuring a very shallow view of how influencer campaigns work.

Awareness and revenue aren’t competing objectives. Any broad reach channel (as influencers should tend to be) generates revenue indirectly, through increasing metrics such as awareness.

Any marketing activity should generate revenue (otherwise what is it there for?), the only real difference is the timeframe that we expect to see this revenue. Do we want to see the impact in days, or weeks or months and years.

Upper funnel activations generate reach and resonance among people who have no relationship with the brand yet. Bigger creators do the heavy lifting here. But the signal that it’s working isn’t engagement rate. It’s brand search demand. Influencer and PR activity consistently drives some of the largest sustained spikes in branded search we see, bigger than paid search or paid social as a long-term demand driver. That brand interest and research behaviour generates revenue, but the revenue may not appear for several weeks or months.

Lower funnel activity aims to turn that brand demand into short term revenue. Short formats, clear product demonstration, partnership ads.

The problem isn’t the structure. It’s that most measurement only ever sees the top or the bottom. The middle - where upper-funnel awareness quietly becomes search intent, then a purchase through a completely different channel - is invisible to most reporting.

We wrote in 2023 that 80% of influencer value will never show up in clicks or codes.

Every model I’ve run since has said the same thing.

For every £1 of revenue you can attribute directly to a creator’s click, another £4 turns up somewhere else in the system. Paid search efficiency. Organic conversion rate. The lifetime value of someone who found you through a creator months before they converted on a completely different channel.

Measure only the click and you understate the real impact by up to 80%.

If you’re only optimising the 20% you can see, you’re managing the business blind to the other 80%.

The most underused lever is what happens when organic meets paid. Virtually every marketing report tries to link each conversion back to an individual channel or campaign, that has never been how marketing actually works. Every conversion is the result of a combination of effects, and how channels overlap and interact with each other is crucial to understanding the true value of each channel.

Whitelisting / Creator Partnership Ads (where the users sees cobranded ads between both the brand and the creator) outperform brand-built assets by 30% on Meta, 40% on TikTok, 25% on YouTube.

That’s creator credibility surviving the move into media spend, instead of getting stripped out of it. Creator content and credibility creating those crucial overlap and synergy effects.

And when done well, those effects can have a huge impact on overall business performance. In the case study below we saw 26% improved paid marketing performance, worth more than £5m in incremental performance across 3 months from the same level of marketing spend.

The knock-on effects compound too. 125% stronger search synergy than business-as-usual advertising. Two-thirds of the people reached are net new, audiences that branded paid ads would never would have been served against. Run creator assets next to standard paid media and video play rates lift 21%, CPA drops 19%.

The potential value is huge, if we measure and optimise to the right things.

So how do we do that?

The instinct is to pick the most “relevant” creator for the category. It’s usually the wrong instinct.

Note, we recently released a new influencer insight paper which digs into how to create effective influencer marketing. It is available here.

On our work for Tangle Teezer, lifestyle, entertainment, family and hair-professional creators generated 14 times more revenue than beauty and wellness creators. The “less relevant” creators outperforming the category creators that everyone would have picked first.

Celebrities are 66% more effective at product launches than they are at retention. Their job is fame, not familiarity.

Micro-creators over-index on conversion relative to their reach, because their credibility lives inside a community, not a following count.

Format matters as much as casting. Video generates up to four times the revenue impact of a static post. Stories will often out-earn Reels or feed content per unit of reach when the objective is conversion.

None of this holds together if you’re still measuring awareness on reach and conversion on last click, because you’ll never see one creating the other. And the simplistic click based metrics act as if only one channel impacts every conversion.

A full-funnel measurement framework has to exist before the campaign starts.

Brand search demand at the top.

Paid media efficiency in the middle.

Direct and indirect revenue at the bottom.

Marketing mix modelling gives us a great consistent measure across all three measurement speeds, which we can compliment and validate with specific measures for each element. This is why we run it all through one system, COmpass, to give both the revenue impact at a contextual business level as well as identifying which activational tactics are working hardest to drive business outcomes.

Awareness creates the synergy which drives conversion. Conversion generates the data that improves the next round of awareness decisions.

Measure, optimise and plan them together and you drive both short term revenue and long term brand demand growth.

FAQs

Should influencer marketing be measured as awareness or conversion?
Neither on its own. Influencer marketing should be measured as one system that produces both immediate revenue and delayed revenue, because splitting it into “awareness” and “conversion” causes brands to measure reach on one side and last-click orders on the other, missing up to 80% of the value the campaign actually creates.

How much influencer revenue is missed by click and code-based attribution?
Up to 80%. For every £1 of revenue directly attributed to a creator’s click, another £4 shows up elsewhere in the business. This additional revenue is generated through paid search efficiency, organic conversion rate, or a purchase made months later on a different channel.

What is the best early indicator that influencer or creator content is working?
Branded search demand. Influencer and PR activity produces some of the largest sustained spikes in branded search of any channel, larger than paid search or paid social, making brand search volume a stronger leading indicator than reach or engagements.

Do creator partnership ads (whitelisting) outperform standard brand ads?
Yes. Creator/brand partnership ads outperform brand-built ad assets by 30% on Meta, 40% on TikTok, and 25% on YouTube, because they carry the creator’s credibility into paid media instead of stripping it out.

What business impact can combining creator content with paid media deliver?
In one Charlie Oscar case study, pairing creator content with paid media drove a 26% improvement in paid marketing performance, worth more than £5 million in incremental revenue across three months from the same media spend.

Are celebrity creators or micro-creators better for driving sales?
It depends on the objective. Celebrity creators are 66% more effective for product launches than for retention, since their value is fame and reach; micro-creators over-index on conversion because their influence comes from community trust rather than audience size.

What should a full-funnel influencer measurement framework include?
Three layers: branded search demand at the top of the funnel, paid media efficiency in the middle, and direct plus indirect revenue at the bottom. Measured together in one system (Charlie Oscar uses its COmpass marketing mix model for this) rather than in isolated awareness and conversion reports.

Subscribe now

Most brands are investing time and resource into AI visibility. They’re seeing their names appear in ChatGPT responses.

They’re reading articles about AEO and GEO and building strategies around citations and structured data.

Hopefully they are even seeing this visibility data go up, but then they look at their P&L. And nothing has changed (yet).

So when do we expect this visibility to turn into revenue?

What’s actually driving AI visibility

Earlier this year we published an analysis of 50 brands across 8 AI platforms. We were trying to understand what determines whether a brand appears prominently in LLM responses.

The findings matter because they tell you where to focus.

63% of AI visibility is driven by long-term brand equity. 22% comes from current marketing spend. 11% from citations and structured content. 4% from PR and influencer.

The dominant driver isn’t your SEO/AEO team’s citation-building programme. It’s the brand you’ve been building for years.

This has a significant implication. Brands that are winning AI visibility right now are predominantly the ones with the strongest offline brand health and the longest track record of consistent marketing investment. Trying to optimise your way into AI visibility through technical content alone is fighting over 11% of the picture.

And it helps to explain why high AI Search visibility isn’t factoring into revenue growth for many brands, because the largest part of AI search visibility is a reflection of the long term brand health.

Is the traffic actually worth anything?

LLM sessions are growing. We see this growth increasing across a majority of brands and sectors.

Of the 5 brands on the chart below (all ecommerce but in different categories) average YoY growth is 253%. Every brand has more than double the volume of tracked sessions in May 2026 than they had in Jan 2025. Three of them have more than 10x the volume.

But even with this growth, LLM-referred sessions are currently less than 1% of total web traffic for most brands. That’s why it doesn’t show up in a P&L read at any normal level of precision.

But those sessions are not equivalent to average traffic. Across the brands we track, LLM-referred sessions are 2x+ more valuable per session than site average. Users arriving from LLM recommendations have already had a conversation about the category. They are further along in their decision-making. They convert differently.

But even at that 2x increased value, the tiny volume of LLM referred sessions still only account for less than 2% of revenue.

The real value comes in the exposure rather than the click through.

At peak AI visibility periods, our modelling shows up to 5% incremental demand generation attributable to AI platforms. Still a very little brother compared to paid search and paid social, but growing and starting to become noticeable.

On the 2x to 10x trend we can see in recent years, that number is likely to only increase across future months and quarters.

The dominant driver of AI visibility is brand equity. Which means brand health tracking isn’t just a soft metric, it’s a leading indicator of future AI performance. Brand health is measurable, tangible and actionable.

Our recent study with Tracksuit looked at the relationships between brand health metrics and search behaviours: every 1% increase in brand consideration drives a +3.5% increase in branded search in the same month. A 1% increase in brand awareness generates a 2.0% organic revenue uplift, typically with a two-month lag.

Brand health is measurable. It moves. And it’s the thing that feeds the training signal and editorial authority that LLMs use to determine which brands are credible voices in a category.

The brands that will win AI search in 2027 are building brand equity now. The conversion will be non-linear: years of brand investment that looks invisible in attribution suddenly surfaces as dominant AI visibility. The brands that don’t track it will only realise the connection in retrospect.

When will the revenue show up?

It probably already is showing up. But at 2%-5% you won’t see it in the aggregate.

If you run models like MMM, then you should be able to isolate the effect.

It will still be quite small but growing, if you are getting 2% of revenue today, then that will likely be 5% by end of 2026, and double that by end of 2027.

The cost of being invisible is increasing.

But your visibility is primarily a reflection of what you were doing anyway. If you were already building brand equity then you likely won’t need to change much.

Track your visibility systematically. If you don’t know where you rank on ChatGPT, Perplexity, and Google AI for your core category queries, you can’t manage the variable.

Don’t over-index on citations. They’re 11% of the driver. Brand investment, organic content depth, and share of voice are the other 89%.

Focus your time, budget and attention on the underlying drivers of the 89% rather than overly obsessing about the 11%.

Connect your brand health tracking to your AI visibility data. The relationship is real and measurable. You should see them moving together, it helps validate the fact that AI visibility is a reflection of brand health not a driver of brand health.

AI visibility will show up clearly in your P&L.

For most brands, it’s already there in the data at a level your current reporting probably can’t see.

The brands that will have the advantage in 2027 who can see the effects and move quickly enough to adjust to an incredibly fast moving landscape.

FAQ

Do we need to do AEO?

AEO (Answer Engine Optimisation) and GEO (Generative Engine Optimisation) are important, but they aren’t the primary driver of AI visibility. Focus on building strong fundamentals in marketing first. The priority is ensuring you have rigorous AI visibility tracking in place so you can understand your current position, and that your brand health measurement is strong enough to see the upstream driver. Specialist execution can follow once you know what you’re optimising for.

Does paid media on AI platforms (like ChatGPT Ads) work?

Too early to say definitively. Most brands are only just starting to test paid ads on these platforms. At Charlie Oscar, we are starting to measure the impact of ChatGPT ads in MMM, but the data is very early. The channel is structurally interesting, high-intent users in a discovery mindset, but we don’t yet have enough data to benchmark incremental ROAS with confidence. Watch this space; we’ll publish findings as we get some more data.

Which AI platform should I prioritise for visibility?

Based on Charlie Oscar multi brand analysis, OpenAI (ChatGPT), and Google AI (Gemini & AI summaries) show the most consistent brand rankings and the strongest impact on revenue. Treat your AI visibility the way you’d treat search, track market share across platforms, not just absolute rank on one.

My brand barely shows up on AI platforms. Is it too late to fix that?

Absolutely not. These rankings change, a lot! But the fix is probably not what you think. If you have strong brand health (awareness, consideration, broad digital presence) and you’re not appearing, the issue is likely content depth and structured data. If your brand health is weak, you’re looking at a longer-term investment. The brands dominating AI visibility right now mostly have one thing in common: years of consistent brand building and high share of organic voice. Focus on building these fundamentals rather than trying to find a growth hack for an ever changing algorithm.

How should I report AI visibility to my board?

Track three things: (1) AI visibility rank across the platforms your customers use, measured monthly (2) LLM-referred sessions and their conversion rate vs site average (3) the gap between your brand’s share of search and its AI platform rank. This gap tells you whether AI is an opportunity or a risk. Present these alongside brand health tracking scores so the board can see the upstream driver, not just the output.

Subscribe now

Useful but wrong is a tough situation to make decisions from.

So how do you make data informed decisions in a world of useful but incorrect information?

How should a brand allocate marketing budget, and what sources of data are most useful?

There are a wide range of tools we all use to justify spend and allocate budget. They can broadly be grouped into four categories: attribution, MMM, incrementality, surveys.

They are all wrong in different ways, at different times. They all have different strengths and weaknesses.

The problem is that most people and businesses have favourites. The vendors who sell each option often promise that theirs is the “single source of truth”.

Let’s be clear, the models aren’t wrong because anyone has done anything wrong. The models are wrong because the real answer is unknowable. Absolute truth of marketing attribution is an impossible task, the human behaviour is too noisy, humans are too irrational and too much of the things which actually influence behaviour are impossible to measure.

But the models are useful because they can signpost us towards the unknowable truth, so we can make better informed decisions.

But those decisions still require nuance and judgement. It is one of the reasons why I believe marketing attribution and strategy will be an area AI will never truly take over. It requires too much human interpretation, AI can improve efficiency of those decisions but are a long way away from automating them.

The real problem is not the data in any of these models, the data is usually correct (within the framework of what it is trying to do).

But correct data in a model with weaknesses still has blind spots.

Most brands pick one or two models, trust it completely, and optimise it into a corner. The problem is how the data is used, not the data provided in the first place.

So in a world of wrong but useful information, how should we actually make those decisions.

Let’s start with the simplest of our data sets:

Attribution Data

The long promoted golden child of digital marketing. Digital marketing will be so much better than anything you have ever done before because we can accurately tell you the exact ad and keyword which drove every sale. And do so in real time.

Sounds amazing, except it doesn’t do any of those things.

Attribution very accurately tells you two things which happened to an individual browser / user / tracking id. But just because the click happened before the sale doesn’t equal “the click drove the sale.”

79% of brands use last-click attribution as a primary measure. Only 22% believe it accurately reflects what drives revenue. I find those numbers shocking. 78% of people don’t believe the numbers, but 57% of them (assuming the 22% who do believe the data are also using it) use it to make their decisions anyway.

More than half.

More than half the people who think the data isn’t accurate are still using it to allocate millions in marketing spend.

There are plenty of numbers which prove this data is incorrect, here is one of my favourites:

The correlation between last-click ROAS and modelled incremental ROAS across the same campaigns: R²=0.03. 3% correlation. Actual business value has a higher correlation to the day of the week or the weather. So how about we just increase spend on hot Wednesdays?

And we can ignore the data and instead apply the sniff test: if your brand search conversion rate is 30% and 90%+ of sales show single-interaction attribution paths. Does that make sense? When is the last time you searched for a brand name that you had never heard of elsewhere? Is it even possible to search for a brand name if you have zero brand awareness driven from something else? It involves some quite lucky guesswork.

So clearly this data is flawed. But it is still useful.

It is the only data we can get in real time with deep granularity.

That keyword level attribution does tell us how people who buy from us are navigating to us, that is useful data. And the trends over time tell us how that navigation is changing.

The causality might not be there, but the correlation is still valuable.

Models and Tests are helpful but not infallible

So if we care about causality then we should look at Marketing Mix Modelling (MMM) and uplift tests (either geo holdout tests or audience control tests).

At Charlie Oscar, we run a lot of these tests and models. Our COmpass models deploy multi layered MMM which looks at paid, organic and external drivers of growth. Our testing frameworks deploy uplift tests alongside paid media activation to build robust validation of results.

These are both designed to help with the causality question, turn something off (or reduce spend) and see what happens to overall data.

MMM solves a lot of the structural problems with attribution. It has a fully view of the context of the wider business, so has a clearly view of the incremental impact of each marketing channel.

But it has its own blind spots: it’s backward-looking, it averages across time periods, and it can only give a certain level of granularity without significant sacrifices to accuracy.

On it’s own it can’t accurately tell you the performance of an individual keyword on Tuesday.

Incrementality testing is the closest thing to ground truth but it’s expensive (in both time and opportunity cost), slow, and can’t cover every channel at once.Typically these tests are run as infrequent validation of other metrics, it is incredibly tough to scale this to provide always on learning at a channel level.

Any granularity below channel is infeasible for a vast majority of brands.

The numbers are often uncomfortable:

When you run these models and incrementality tests, the numbers are often humbling and frequently uncomfortable.

Channels which have received significant levels of investment over years, suddenly show up with much weaker performance than has been presented back to the business.

Our COmpass data consistently shows us where brands over invest in low incrementality channels:

• Brand search: <10% incremental (users were going to search anyway)

• Retargeting: very low incremental value

• Coupon and discount affiliates: <20% incremental

• Meanwhile some channels which were paused or scaled down due to poor click based ROAS were likely driving significant unmeasured value for the business.

A big cause of why 57% of brands are continuing to use data they don’t believe is that they can’t face the pain of reassessing their knowledge of past successes and failures.

Time makes this even harder:

Budget allocation isn’t just about where you spend. It’s about when you read the results.

A travel brand running January spend showed a ROAS of $2 in-month. By December, the same spend had generated $5 of revenue, once adstock and delayed consumer journeys were accounted for.

Both attribution and incrementality tests (which frequently do not run longer than 1-2 months) would have shown the campaign with a weak impact because they cut off impact before a majority of the revenue is generated.

A June priming campaign showed in-month ROAS of $3.27. By August peak, that number was $7.52. But by the time of peak a large amount of those purchases would have been reattributed to the navigational retargeting and promotional ads which directly preceded the purchase.

So how do you actually make decisions in a world of confusing data?

Accept that you need multiple models for multiple decisions. Not one number that answers everything.

Accept that every method has weaknesses, aim to have at least two sources of truth for every important decision. When they disagree will often tell you as much as when they agree.

Use the right method for the right decision speed.

Some decisions need to be made fast, attribution data is the only real option for real time decision making. In these cases we often have to sacrifice accuracy for speed. For example, non brand keyword optimisation which doesn’t need a contextually accurate and causal measure to be able to optimise towards the strongest performing keyword in real time.

Some decisions need to be considered and as accurate as possible, and we can sacrifice some speed and granularity to get that contextual and casual data. For example, how much should we spend on CTV ahead of peak seasonality. We have time to make that decision and should use that time to lean into more casual methods such as incrementality testing and MMM.

Use MMM for annual and quarterly strategic allocation. Use MMM outputs to inform and adjust real time attribution measures.

Use incrementality to validate faster moving measures.

Use attribution for daily and real time in-platform efficiency.

Use brand tracking (and post purchase surveys) to validate the changes in behaviour against the changes in how you are perceived by audiences.

The best budget allocation process isn’t the one with the best model. It’s the one where every level of the organisation is making decisions with appropriate evidence for the timescale they control.

Any business who tries to make decisions from any one of these wrong but useful models, will fail on at least one of their decision speeds.

“All models are wrong, some are useful”. The goal isn’t to find the perfect model for all cases.

A swiss army knife is useful for a wide range of tasks, but you won’t find one in any specialist workshop. Because they will fit the specific tool for specific need.

Charlie Oscar measurement, via COmpass, is built on this principle - combining attribution, MMM and incrementality signals so brands can make decisions at every speed without having to choose between them.

We need to understand what each model is actually measuring, use it for that purpose only, and triangulate to a decision you can defend.

FAQ

If last-click is so bad, why do so many brands still use it?

Because it’s easy to implement, universally available in every ad platform, and produces a clear number. The problem isn’t that marketers don’t know it’s flawed, it’s that it’s the default, and replacing defaults requires organisational change, not just better data.

How much budget should go to brand vs performance?

There’s no universal answer, but MMM consistently recommends higher brand investment than attribution models suggest (often 40–60% of total budget for established brands). The right split depends on category, lifecycle stage, and competitive pressure. The wrong answer is letting your attribution model decide, because it can only see demand capture, not demand creation.

Does MMM replace incrementality testing?

No, they answer different questions. MMM tells you the historical contribution of each channel to revenue, averaged over time. Incrementality testing tells you the causal effect of a specific channel in a specific period. MMM gives you strategic direction; incrementality gives you a reality check on whether the model’s assumptions hold.

What’s the minimum viable measurement setup for a brand that can’t afford full MMM?

Start with two things: (1) a clean incrementality test on your highest-spend channel, so you know if it’s actually working; and (2) brand tracking that measures awareness, consideration, and preference because these are the leading indicators of future commercial performance that attribution will never show you. COmpass is designed to build on this learning, bring the rigour and insight of MMM without the cost and time requirement of legacy econometrics.

Subscribe now

In this article, we look through three separate measurement approaches. Three completely different data sources, across a wide variety of brands and categories.

But we consistently see the same thing:

The impact of a brand advertising campaign returns to baseline in roughly two to three months.

We know it from COmpass, Charlie Oscar’s marketing mix modelling platform, which measures the commercial impact of marketing spend.

We know it from independent brand tracking studies run across multiple brands and clients.

And we can see it in the search demand on 105 Super Bowl ad campaigns run between 2005 and 2025.

The number that keeps appearing across all three: around 11 weeks.

That is the average window before a campaign’s measurable effect on consumer behaviour, brand perception, and commercial return either fades significantly or disappears entirely.

And that is clear even on the biggest brand moments in marketing. The most notable Super Bowl campaigns, costing upwards of $250k per second, generate short, sharp brand spikes which disappear in weeks not months.

Engaging content generates commercial impact

Let’s look at the positives first. Brand advertising campaigns genuinely move the numbers that matter. Brand awareness, consideration and investigation increases, share of search improves and organic attributed revenue increases.

We measure this in three ways: brand awareness tracking (what people think), search demand (what they do) and organic revenue from MMM (what it’s worth commercially)

These three metrics give us three independent measures of how brand activity impacts business performance. What people believe, how they behave and how that translates into revenue.

The top-rated Super Bowl ads generate real search demand. Little Caesars hit a Google search index of 99 in the week of Super Bowl LIX, against a pre-game baseline in the low 60s. Budweiser’s Puppy Love ad in 2014 spiked to 39 against a baseline of roughly 3. Hyundai’s Smaht Pahk drove sustained search interest well past game day.

Brand tracking data shows the same impact from brand campaigns. Awareness metrics move in the weeks after a major brand campaign activates, sometimes relatively small 5-10% shifts, sometimes much larger 15%-20% increases. Consideration, Brand intent and research scores tend to follow as we see the impact of increased brand activity.

COmpass (Marketing Mix Modelling) confirms that these impacts drive incremental revenue. Every channel we model delivers a real commercial return. Organic baseline revenue increases 20% for every 10% increase in brand awareness. Channels such as TV, OOH consistently show strong revenue payback over the weeks and months after activation.

The impact of brand activity on business performance has been proven consistently, and is clear on all three of our core metrics.

How long does a brand advertising campaign actually last?

The problem isn’t the impact, it is the duration of the impact.

Consistency beats momentary brilliance.

From Google Trends: The average time for search interest to return to its pre-campaign baseline across 105 Super Bowl ad campaigns is 2.76 months. $8m for a 30 second ad, for a brand impact which disappears within 3 months. Ouch.

From Brand Tracking: Brand tracking data shows brand awareness metrics (aided awareness, consideration, brand familiarity) typically returning to pre-campaign levels within eight to twelve weeks of the campaign going dark. The brand campaign worked, it just doesn’t hold the uplift without continued investment.

From COmpass MMM: Even TV and OOH campaigns (the channels with the longest tail effect in most attribution model) deliver 79% of their total commercial impact within two months of running. Search and shopping delivers 100% of its commercial impact within two months. Paid social: 95%. Paid video: 85%.

The most patient channel in the mix, TV, still generates four-fifths of its measurable revenue within the same two-month window that brand tracking and search interest data point to.

All three methods show we get a strong business impact from brand activity.

The effect wears off. Usually within 3 months.

Certainly short enough to question the 1 or 2 brand campaign spikes per year that many brands plan around.

Beating the decay.

The average Google Trends return time is increasing. The early era of our data (2005–2014) averaged 2.61 months. The most recent decade (2015–2025) averages 2.90 months.

The most likely explanation is that these bursts are being amplified and extended through digital paid and earned media. A Super Bowl ad in 2006 lived for a news cycle and then disappeared. A Super Bowl ad in 2025 gets written about, shared, dissected on TikTok, and surfaces in recommendation algorithms for months. This extends the impact, but the uplift is still remarkably short.

47 brands in our Superbowl data pool showed no significant search spike during Superbowl activity (including Budweiser, Amazon, Chevrolet). Not because their ads weren’t effective, but because they had long sustained brand investment over time which limited the size of the spike but also countered the decay in brand interest.

What can we do about it?

When three independent metrics all tell the same story, it becomes compelling.

A short term brand campaign burst can look great, but it can’t grow sustainable business impact. The uplifts which disappear in 3 months need to be replaced, or ideally sustained over time with consistent brand investment.

The brands maintaining elevated search interest, sustained brand tracking metrics, and compound commercial return are not the ones with the biggest campaign moments. They are the ones running consistent marketing programmes that the campaign amplifies.

The channels that consistently sustain the baseline between campaign bursts are those with long, indirect effects along with consistent always on delivery, influencer marketing is particularly valuable for this purpose.

Our own COmpass data shows that 80% of influencer-driven revenue impact is indirect, showing up later in search, organic and other channels rather than as an immediate click. Run consistently, influencer activity keeps the baseline elevated rather than letting it decay back to zero between campaigns.

Don’t build the spike, build the baseline.

FAQ:

How do brands measure brand campaign effectiveness?

There are a number of ways to measure brand campaign impact. Including brand tracking surveys, geo uplift studies and Marketing Mix Modelling (COmpass by Charlie Oscar) . Charlie Oscar combines these measures to give a rounded view of overall brand campaign impact across both direct (impact on organic revenue) and indirect (improvement in performance channel effectiveness) impact.

How long does a brand advertising campaign last?

This always varies by campaign, by brand, by category and is highly dependent on execution. But the effect wears away faster than you probably expect. From our data across a variety of brands, using different measurement approaches shows that almost all of the effect decays within 3 months.

What is the marketing decay rate for brand campaigns?

Marketing decay rate shows how long the effects of a campaign affect a business. MMM (Marketing Mix Modelling) is a good way to measure these effects. The decay rate varies by campaign and channel, but for a majority of brand campaign executions the business effect will decay after 3 months.

How can brands sustain the impact of a campaign over time?

Brands should aim for always on executions, rather than one off brand campaign spikes. Brand campaign spikes can drive short term uplifts, but these tend to decay after 3 months. Always on brand activity drives long term sustainable brand growth which helps to mitigate these spikes and decay rates.

What does Marketing Mix Modelling show about brand campaign effectiveness?

Marketing Mix Modelling (MMM) is a good method for measuring brand campaign effectiveness. MMM consistently shows that brand campaigns improve organic revenue growth for brands as well as improving the efficiency of performance channels.

Why do brand uplifts disappear after a brand campaign ends?

Brand campaign uplifts decay over time as potential customers become less exposed to the brand advertising. Brand advertising drives business value by driving increased mental availability (how often a customer thinks of your brand when looking for a product or service), but this mental availability reduces if customers are less exposed to your brand.

Subscribe now

Most businesses either try to use the same metrics across all three (which is virtually impossible)

Or use completely different metrics for each of the three. Which makes these metrics a terrible choice for at least one of the key decision points.

We can broadly break these decisions down into three speeds:

1. Strategic decision making: Decisions made slowly, with a lot of thought and consideration. Setting the direction, defining priorities and making significant bets on overall growth and trajectory. Most businesses do this quarterly or twice per year.

2. Budget allocation decisions: Moving spend between marketing channels, across tactics and creative messages. These decisions should take place within the constraints of the strategic direction (but this doesn’t always happen.) For most businesses this is once or twice per week.

3. Optimisation decisions: For a long time, this was aligned to the budget allocation decisions, but now this rarely involves a human. Platforms shift spend between campaigns, ad groups and creative messages in real time. Based on the direction given in platform setup and targets, but there is a significant nuance in getting these settings right.

As you move from the strategic decision making to the optimisation decisions, we see two trends. The speed increases, and the human judgement reduces.

And the people involved changes a lot. Most businesses will not have a single person who touches all three of these decisions. Rarely will a single person be involved in more than one of them.

That creates a problem. Because the strategy will only work if the day to day decisions are deploying it. And the day to day decisions can only deploy it if they are using the same measure of success.

This is where most businesses fail.

We can see this in the measures, KPIs and metrics used across each of these decisions.

1. Strategic decisions: MMM has held a strong place in these decisions for decades. Speed isn’t really a constraint. Breadth and business awareness is crucial, depth of channel insight is less important. So this leans into the strengths of MMM, and somewhat mitigates its weaknesses. Key decisions are made on the incremental long term value of marketing.

2. Budget allocation: The methods differ wildly across businesses, but most businesses will have some kind of third party measurement which aligns performance across multiple platforms and campaigns. Usually these run on a form of digital attribution, but increasingly are incorporating some of the learnings from MMM or uplift testing. But usually even when MMM is incorporated it will be a different form of MMM measurement than the one used to set the strategic direction (eg media mix modelling which ignores pricing, competitors and organic marketing sources.)

3. Optimisation decisions: Decisions made exclusively by the platforms, based on some preset targets and objectives. Usually some combination of maximising platform tracked conversions or revenue, or maximising landing page views or reach metrics. Each platform does it differently and has no visibility of data outside it’s platform.

There is a fundamental difference in the data and KPIs being used to make decisions at each speed.

Even when each team is making the best decisions informed by their data, there is a huge disconnect between the strategic direction and the optimisation happening across channels and within a platform.

On a tangible example:

A business sets its strategic direction based on an MMM. The model tells them that brand investment is generating 40% of their total revenue. That TV and outdoor are responsible for 60% of that brand value. And that their paid search campaigns are predominantly capturing demand that would have converted anyway, so they are not creating new demand.

That’s solid strategic direction based on some robust, contextual data.

Now watch what happens when that direction filters down to the weekly, daily and real time decisions.

The team making budget allocation decisions is working off a third-party attribution tool. That tool doesn’t know what the MMM knows. It can’t see TV spend. It has no concept of competitor activity. It has no view of the brand’s long-term trajectory.

It sees search campaigns converting at a strong ROAS and social campaigns converting at a weaker one. So it recommends moving money from social into search. So the team making the decisions are left with a problem, do I follow the strategic direction and ignore my reporting platform? Or do I follow my attribution and deviate from the strategic direction?

And in-platform, the algorithm is doing exactly the same thing. Optimising to the conversion signal. Finding more people who look like recent converters. Building its own internal model of success, entirely defined by what it can see inside its own walls. It is trying to increase ROAS, and using the data it has available to do that.

Three rational decisions. Three different data sets. Three completely different directions.

The data problem

This is what makes it so hard to fix. It isn’t about people making bad decisions. It’s about people making reasonable decisions from fundamentally incomplete pictures.

The MMM has the most complete view of the business. It captures all media spend, pricing and promotions, competitor activity, macroeconomic conditions, seasonality, organic search trends, and the full revenue effect across long time horizons. It is built specifically to separate these interacting factors and give a clean read on what is actually driving performance.

Budget allocation tools see a much narrower version. Typically your own digital spend and some cross-channel attribution logic that tries to join the dots between them. No competitors. No pricing context. No delayed effect of brand investment. And no clean way to separate the organic demand that would have existed regardless of any advertising.

In-platform tools see almost none of it. Their own impressions. Their own clicks. Their own tracked conversions. They are optimising to a signal that represents roughly 15% of the data the MMM uses to make decisions.

The problem isn’t that any of these methods are wrong. It’s that they’re looking at different things. And when the decisions they drive are pointing in different directions, the strategy never reaches the execution layer.

Using one method across all three isn’t the answer

The obvious solution sounds like: pick one method and use it consistently.

It doesn’t work.

MMM is built for strategic decisions. It takes weeks to build, needs months of data to calibrate, and produces insights at a level of aggregation that is useful for setting direction. It has the breadth and the business context. It doesn’t have the speed or the granularity to tell you whether to increase the budget on a specific campaign this Tuesday.

Attribution-based tools are built for the middle layer. Faster, more granular, more actionable at the campaign level. But they systematically overweight the bottom of the funnel, miss the indirect effects of brand, and have no visibility of the business context that the strategy was built on. If you use them to set strategic direction, you will consistently underinvest in the channels that are actually driving your business.

In-platform optimisation has its place too. When the targets and objectives are set correctly, platforms are genuinely effective at finding the right audience within the right budget.

Three methods. Three jobs. Three different things they were built for.

So our job becomes managing the gaps between them.

There is a way to create a consistent line between the different speeds.

It starts with the MMM. The strategic layer. This tells us what is actually driving revenue at the business level including the things that attribution will never see.

But we need to get those insights out of the board room and into the other decision layers.

Specifically, we use the MMM to correct the known biases in the budget allocation model. If the MMM shows that paid social is generating twice the long-term revenue impact that the attribution tool suggests, we adjust the values used in budget allocation accordingly. We’re not replacing one model with the other. We’re using the more complete model to fix the blind spots in the faster one. We are building solutions here which build weekly MMM outputs on incomplete data but informed by the robust strategic models. It mitigates the tradeoff between speed and accuracy.

That calibrated media mix model then informs how we set targets in platform. Rather than telling the algorithm to maximise tracked conversions, we use the media mix model to establish what real revenue value looks like, channel by channel. We build those values into the bidding strategy and value rules. So rather than every campaign getting a ROAS target of £4, every campaign gets a unique ROAS target based on the MMM findings, which changes with respect to seasonality, brand activation, pricing and promotions.

The result: when the platform optimises, it’s doing so against a target that reflects the strategic direction. The strategy reaches the execution layer.

There is no single source of truth. And that’s fine.

It would be great if we could have one single source of truth to do a great job at all three speeds, but it doesn’t exist. The tradeoffs are too big.

Strategic decisions need breadth and business context. Budget allocation needs speed and channel granularity. Optimisation needs real-time signals that can act without human involvement. These are genuinely different requirements and they need different methods.

What they don’t need is different directions.

The MMM tells you where to go. The media mix model decides which roads to take. The platform algorithm navigates the traffic. When all three are working from the same map and compass, you stop losing value at every handoff.

The goal isn’t a single source of truth. It’s a consistent line between them.

Subscribe now

I recently spoke at Meta’s creators event sharing some of our research into the business impact of creator content in marketing.

One of the most common questions is “what type of creators should we work with?”

My answer is the most valuable creators are “probably not the ones you were thinking of”.

So let’s get into the data behind that answer.

Firstly, how do we define “most valuable”.

I measure creators on the revenue impact they have on the business, but not through click attributed revenue or voucher code redemptions.

Clicks and codes account for around 20% of the real revenue contribution of creators and influencers. 80% of the value is indirect, driven through the broadcast effects of people seeing content which they resonate with then acting differently in the future.

It is a really consistent finding whenever we measure real influencer and creator value for brands.

We can measure both the direct and the indirect effect of influencer and creator content through MMM, so all the numbers I share in this article come from those MMM studies.

The revenue impact of creators and influencers can be massive, and it comes from two key factors:

1. The broadcast value of the organic reach from influencers

2. The impact of influencer and creator content on paid campaigns

Both are significant drivers of revenue, the creator (whitelisting) content impact on paid channels can be up to 30-40% CPA improvements.

But the best performing content and creators on paid whitelisting are not necessarily the same as the creators and influencers who drive the strongest organic value.

If we look at the value of the organic reach.

Bigger isn’t better.

Clearly celebrities will get bigger reach than smaller influencers. But when you take the equivalent investment and group up micro and macro influencers then the difference in reach can start to be mitigated (30x micro influencers with an average following of 20k will frequently deliver larger reach than an influencer with 1m followers)

But even when they do deliver larger reach, the revenue value rarely falls in line.

In the study above, celebrities represented 40% of total reach, but only 30% of response (engagement and reactions) and 22% of revenue uplift.

Meanwhile the Micro and Macro creators delivered a combined 51% of revenue uplift from their combined 30% of reach.

So if the most valuable creators don’t need to be famous.

Do they need to be relevant?

Not really. At least not in the way most people talk about relevance.

This chart is from some analysis for a fashion brand. They activated across a wide range of creators and influencers.

The most category-relevant are the fashion influencers.

Fashion influencers are very relevant for a fashion brand. Obviously.

But we can rate their relative revenue impact through revenue per reach (revenue driven / reach).

The higher the bar, then the more revenue we get for the same amount of reach with each influencer type.

The fashion influencers fall down to the bottom.

Very category relevant, relatively weak at driving incremental revenue.

The ones at the top: Sport and Family.

Sports people as creators for a fashion brand. Not very relevant. Very effective.

(This is not a sports apparel brand btw, and demographically it skews younger and female.)

Also a strong performance from entertainment. Again, not very relevant but very effective.

The thing that all these influencer groups had in common was they spoke very differently to the brand and to the “relevant” fashion influencers. So they attracted a new audience, with a new message, and created incremental revenue impacts.

So you want to work with a range of sizes of influencers and a range of categories and niches.

But as with most things, it isn’t quite that simple.

The “best” influencers aren’t the best all year round, and for all objectives.

If you want to drive new product launches then the celebrities can help make a bigger bang, that shows in the data.

If you want to create loyalty then long term mid sized creators help encourage that behaviour.

If you need to promote sales, then most things work as long as you are getting a net new audience.

If you want to drive market leading results, take a few risks.

Work with the people that you might have initially rejected.

Push the brand guidelines a bit.

And let them tell their story in their own way.

A sports person will speak about fashion in a very different way to a fashion influencer.

Subscribe now

The trackable part was kind of true, the stronger business outcomes unfortunately didn’t always follow.

Click tracking is the worst thing that ever happened to marketing. It took attention away from the behaviours that actually mattered.

Marketing clicks are an area where Goodhart’s law is particularly true: “When a measure becomes a target, it ceases to be a good measure”

I spent 15 years of my career at marketing agencies telling brands how crucial it is to understand every granular response to marketing and to optimise to the real time click data flowing through your marketing campaigns.

It took most of those 15 years for me to realise that we were optimising to the target and removing the value of it as a measure.

Years of running campaigns which had ever improving click tracked revenue, lower CPAs, higher ROAS. And years of meetings with brands who saw all our campaign charts going up and to the right, while their business performance was flat.

Now I wish I could rewind that time and remove the clicks from every report, from every meeting and from every decision.

Firstly clicks as a measure is relatively meaningless:

• Most people never click ads. More than 4 in 5 people say they would never click on online ads. Source (this research is from back in 2009, I suspect the current numbers are significantly worse)

• The ones who do click aren’t representative. And 85% of clicks come from 8% of users.

• Most of your marketing impact is blind to clicks. If you hit an optimistic 0.5% click through rate from online ads, then for every hundred people who clicks you have also served advertising to 20,000 people who didn’t. You have no idea if those people liked what they saw and or actively hated it.

The two things that should stand out on this table is the concentration of clicks into a small pocket of users, and the direction of travel on all the key metrics: it got worse 2 years later.

A majority of clicks come from a tiny amount of users, and it 2009 was worse than 2007.

I can’t find a more recent version of this study (it looks like nobody wants to replicate it…), but all the data we have suggests these trends have continued and accelerated in recent years.

So maybe click data is just meaningless.

If only.

The data I have shows making decisions on clicks actively make your decisions worse.

• 80% of business value from marketing is indirect, it will never be measured by clicks or code redemptions. That is consistent across hundreds of models we build and businesses we measure.

• A huge part of that is because value is delivered over a longer timeframe than most marketing reports account for. Clicks are measured in real time, value is delivered over days, weeks and months.

• You can see that on the average payback period by channel here

Even for channels such as Search and Shopping, only 60% of the total value is delivered in week 1. So the campaigns and keywords which look best on click ROI at the end of every day are not necessarily the same as the ones which drive the strongest long term value.

When we look at channels such as Paid Social, Paid Video and Influencers the first week value is tiny. Who really wants to make their decisions based on an unrepresentative 17% of the data?

If we act on this data we would try to funnel as much spend as possible into Search and Shopping campaigns. There are numerous studies showing the negative impact of that (I have shared some in previous articles.)

When we look at those engagement based channels, the data is even more clear.

This chart looks at the correlation between a series of platform based metrics, and the incremental revenue generated from the channel (measured through MMM.)

The higher the bar is, then the closer that metric correlates to actual business value.

(Note that this is correlated to value, shares and reactions aren’t driving the value but they are an effective measure. Until they become a target…See Goodhart)

Look at the CTR bars. Across all platforms, CTR correlation is weak at best, frequently non-existent.

Driving better Click Through Rate does not drive better revenue impact on these channels.

In 4 out of 5 of the platforms CTR is a worse predictor of success than CPM.

CPM purely measures how cheap our inventory is, how much reach we get for our money, but it is a better predictor of real business value than clicks.

The best predictors are engagement metrics which reflect how much someone cared about the advertising. View rates, Reactions, Shares, Saves and Comments. These are valuable measures because they are rarely used as targets.

And because they are not used as targets they continue to reflect value as a measure.

If you made your creative optimisation decisions on CTR (like a majority of brands still do), then your business performance will decline.

I wish I could roll back the clock and stop the clicks ever getting into all the marketing reports, but as that isn’t an option the best we can do is try to reverse Goodhart’s law.

Stop making anything related to clicks the target.

Stop making decisions on it.

Stop even looking at those numbers when you are making decisions.

Make your decisions on the long term (incremental) value seen from all channels, on both direct and indirect impacts.

Make your decisions on the data which shows how 100% of your audience cares, rather than how a mis representative 0.5% act.

Subscribe now

LTV vs CAC.

How much you earn off a customer vs how much it costs to acquire a customer.

In most businesses, the marketing and acquisition teams try to reduce CAC.

While the retention (customer relations, email etc) teams try to increase LTV.

But as with most things, it isn’t quite that simple.

The type of customers we acquire, and how we acquire them has a significant impact on how we need to treat them in retention, and the expected value we can see from them over their lifespan.

How and who we acquire directly impacts LTV.

Plenty of businesses find themselves with disagreement between what is causing LTV to drop. Are retention policies performing worse? Or is it a deteriorating quality of acquired customers?

Fortunately we can run models and analytics to help isolate these changes down to the underlying causes.

Moneyball (the baseball film featuring Brad Pitt and Jonah Hill) is famous for applying data models to sports and predicting future performance. The main theme in the film is based around using data to find undervalued players, and building out the best team for current performance.

But one of the best applications of these models in modern sport is through identifying young talent which has potential to perform in the future. In English Football, the likes of Brentford and Brighton are exceptional at this and their data analytics allows them to compete (and beat) teams with significantly more resources than them.

Their models (both based on their owners’ learnings through betting and gambling data models) let them use current player behaviours (note behaviours not outcomes; so shot creation not assists, big chances not goals) and match to the career trajectory of other players who looked the same as a similar age and development stage.

It is very similar to how we identify future value in newly acquired customers.

We want to know how they act, not just how much they spend, then find similar customer profiles.

This approach gives us a much better understanding of what really causes (and limits) customer LTV.

It is significantly more accurate and insightful than simplifying applying multipliers on to 1, 3 or 6 month revenue values.

It means we can look for the footballers who play in a particular way, that we expect to develop in a particular way, rather than just who scored the most goals at aged 17.

We can run these predictions through cross sectional audience models, which identify customers within four key buckets of behaviours:

1. Product - What are they buying, and in what combinations? Did they only buy top selling SKU, or did they buy a bundle of products?

2. Audience - Who are they are (demographically)? Where do they come from (geographically)?

3. Promotion - How did they come to us? Were they acquired on full price or on heavy initial discounts?

4. Marketing - Where did they come from? Which channel navigated their interest and through which message?

Each of these buckets can have dozens of individual data points, and the potential combinations for every customer easily reaches into millions.

Each of the data points have inter-correlations between each other (eg customers acquired on email might be more likely to have larger discounts, or customers in London are more likely to buy Product X vs Product Y.)

So we can’t run this type of analysis through simple customer cohorts on pivot tables. We need to try to separate the inter-correlations.

These cross sectional audience models help us to do that.

We can model out the impact of each individual data point on predicting longer term value, from our longer serving, more developed customers.

Then match this back to our new customers aligned to “what they both looked like at the same development stage”.

Every brand will get different results from these models, but there are some consistent findings.

There is significant impact on LTV from factors owned and controlled by acquisition:

This shows an anonymised version of these predictive models for an ecommerce brand, there are 19 key metrics which statistically predict future customer value (grouped in 14 in this visual output as some are breakouts of Product types and audience characteristics.)

The index values and colours indicate the relative strength of signal, not the amount of revenue predicted. For example, for this brand predicted value increases with higher customer age, but the index value is below average at 78 as it is a weaker signal than the other featured metrics.

The strongest predictors of long term customer value (18 month LTV) are highlighted in green.

• Product Bundle and Basket value : It is unlikely to surprise anyone, what someone spends on first order is a strong predictor of long term spending power.

• Reactivated customer: A really strong predictor of value, winning back a customer who previously lapsed is a very strong signal of value (usually very positive impact on lifetime value.)

• Discount % and Limited time discounts: Really strong predictors of lifetime value, but usually a negative impact. If you acquire a customer on strong and short lived discounts then it will tend to help improve CAC but at the expense of LTV.

We also have some strong impacts from areas which marketing and acquisition can directly impact:

• Location: It probably won;t surprise you that customers in wealthier areas tend to be stronger LTV customers, but that is rarely factored into acquisition strategy. These customers will frequently cost more to acquire, but can frequently payback more than this in overall customer economics (even after accounting for differences in product selection and discount usage.)

• Channel and Message: When we acquire on a short term message on a short term product need, then we acquire less loyal customers than if we acquire someone who researched and bought into us as a brand.

What does that look like in real data?

We can see that product selection (and first basket value) predicts 65% of first month value. As we would expect, this first order is by far the most important factor in first month revenue.

The other factors are significant, but relatively unimportant compared to people who buy the most expensive products.

But when we extend out to 18 month value, this first order product selection and basket value reduces down to 31% of prediction power. Still important, but less than half as much as it would previously appear.

On this longer timeframe we see the importance of the type of customers acquired through audience metrics and marketing channels. And particularly the tradeoff between promotional impact on short term CAC and long term LTV.

51% of long term customer value is directly impacted by choices we can make at the point of customer acquisition. 24% through the marketing channels we use to acquire customers and the messages we use to acquire them.

27% by how hard we push promotional levers we put into the market, and how we tradeoff long term customer loyalty against short term promotional acquisition.

When we assess customer long term value as a lever controlled by both retention teams and acquisition teams, we can start to outperform the market and potentially punch above our weight by better selecting the best type of customers.

Subscribe now

Even the ones who have a really good understanding of lagged impact of media and media carryover effects.

Carryover effects are the adstock impact you see through attribution such as Marketing Mix Modelling (MMM). The marketing we put out today impacts today, but it also impacts tomorrow, and the day after etc.

MMM (and some other measures) can measure these effects, but it is also just good common sense. Some of the people who see, care about and respond to our marketing arent going to buy today, or this week, or even this month. Maybe they are too busy, maybe they are waiting for a current product to wear out or a subscription to run out, maybe they just don’t have any spare cash until next pay day.

That carryover effect is one of the things I am yet to meet anyone who disagrees with. Yet most businesses ignore it in how they measure impact and make decisions.

It is really evident in Monthly Marketing Reports.

We increased budget on YouTube this month, ROAS decreased. Obviously.

This month has 100% of the cost allocated to it, but potentially only 30% of impact has happened yet.

So the data ignores the carryover effect that basically everyone agrees exists.

This has a huge impact when we look at seasonal businesses. And virtually every business is seasonal. Whether it is spikes in revenue due to Black Friday or Holiday sales, or a natural travel seasonality

Priming customers before sales and seasonality peaks is absolutely crucial for any brand. And it is under invested by a majority of brands.

The inclination is to spend more during the peak months. ROAS is higher, demand is higher. But competition is also higher and increasing spend only in these months will max out returns much quicker.

When we measure and optimise spend profile including these carryover effects, then it pushes us towards prioritising priming customers in these pre-sale periods.

The carryover length and strength varies by channel, but there are key channels which can significantly improve this peak performance by investing pre-peak.

We can see a key example for a travel brand investing during and pre summer peak.

There are two key seasonal peaks in Jan-Feb and Jul-Aug, and the in month driven revenue in these months is very strong.

But there is also a very strong contribution from compounded revenue from previous months marketing investment.

This is even more clear when we look at the revenue driven from long term payback channels (such as YouTube and CTV) in January, and the effective ROAS as this compounded revenue is generated.

The whole of the $2m spend investment is seen in January.

In January we see $4.1m in driven revenue, giving a moderate ROAS of $2 of revenue driven per $1 spent.

After accounting for effective margin this makes the investment appear barely profitable.

But we know that the 31st of January isn’t a hard cutoff for revenue impact (despite it appearing that way on our marketing reports). And the spend investment continues to deliver revenue across the rest of the year, particularly in those Jul and Aug seasonal peaks.

As the compounded revenue continues to accumulate, the ROAS for our January spend investment increases to $4 by July and $5 by December.

Suddenly the moderately profitable return on in month revenue increases to a strong and very profitable investment with this compounding revenue effect.

When we look at investments made in June, directly before this peak seasonality, we see how strong these priming effects can be at driving compounded revenue.

With the strong category seasonality in July and August, the June investment actually delivers more revenue in July and August than it does in June.

And the in month ROAS of $3.27 shoots up to $7.52 by end of August, and past $9 by end of November.

The June priming activity delivers incredible performance, but a majority of that will never show up in a month by month report.

And investing ahead of key seasonality peaks allows us to compliment the in month marketing investment with significant lagged, compounded revenue from previous months.

Subscribe now

The one true way to show the impact of what we do. Turn something off in one place or for one group of users and see the difference in business performance. Like science class back in school.

No attribution logic, no methodology.

An unarguable measurement.

Except it isn’t.

Unfortunately marketing doesn’t happen in a science lab. There are a lot of things we can’t control, and suddenly our uplift tests stop being “the only thing that changed was this” to “well it looks positive but we can’t be sure”

Today I wanted to share a couple of recent examples which show some of these challenges. And why that means that I consider A/B testing to be another tool in our toolkit rather than the absolute truth measure.

First let’s take a geo uplift test for a retailer.

They operate across the UK, and we wanted to stress test our MMM outputs for measuring OOH activity.

So we set up a test to keep all other channel investments the same, and only run OOH in two cities. Those cities represent % of revenue so it should give us a good uplift test result.

(No prizes for guessing the biggest City contribution here, but you can see strong contributions from Cities 2 and 3, and a big longtail across the wider country.)

And revenue for those cities aligns with overall country level performance for previous weeks, so it looks like they are representative of overall behaviour.

So a 4 week test, OOH running in two cities and not in the other top 8 cities. All other channels are trying to stay consistent (this is tough but we did our best), no other changes in pricing, promotions etc.

Measuring the impacts after 4,8 and 12 weeks. As we expect OOH to have a lagged impact on revenue.

% uplift in revenue in our OOH cities. And no uplift in our control cities.

A huge success. Statistically significant uplift when you run it through the tests.

Except for one problem.

The OOH campaign never ran. I just made it up.

All the city comparison data is real. Those uplifts happened. Those cities overperformed.

But the OOH campaign never existed.

So how can something that sounds so scientific and passes all the stats tests give us fictional results?

It comes back to “everything else was constant”.

There are quite a few problems with that assumption.

There are several impacts on revenue that we don’t control, and have never been able to control.

Competitor activity, organic press coverage, word of mouth, weather, random luck and white noise. All really important for driving revenue outcomes, and we have virtually no control over any of them.

We also have much less control over the paid marketing distribution that we used to. If you are running campaigns correctly you shouldn’t have too much control, you need to let the algorithms optimise.

But they will optimise towards short term success, and they won’t try to keep your distribution even. The algorithms will chase success, so they will spend more in pockets where they see success.

That particular city uplift example, if you dig into the data then the spike is because of an increase in proportion of paid social traffic skewing towards those cities (not targeted, just algorithms doing what they are meant to do and chasing performance) and a spike in referral and organic traffic which seems to be linked to some particularly successful press coverage which outperformed in one of our exposed cities.

The two things are likely linked because the paid social algorithms will chase the positive signals of the press coverage.

Those changes were only evident when running some post analysis and looking for other causes. Something that is rarely done when a test shows positive uplift. And only possible with a bit of human inquisitiveness and cynicism.

Otherwise it would have been easy to convince ourselves that fictional bus stop ads performed brilliantly.

The opposite could also be true. Some of our control cities could have seen the external spike and it would have made the successful uplift from OOH appear non-effective.

A second example looks at user uplift tests.

This is a site test on the impact of pricing.

A really clean control vs exposed test. 25% of users got a price 10% lower than current. The other 75% got the current price.

Test results were clear, lower price increases conversion rate by more than 10% and we get a net win on revenue and margin. Again it passes every statistical significance test.

Brilliant. So we implemented new pricing site wide.

Now site wide. That conversion rate uplift disappeared.

The uplift in revenue and margin never showed up when the test finding was applied to the real world.

Why did it not show up? I still don’t know. The test was run correctly, the findings were real.

Maybe the test ran at a time that customers were a bit more price sensitive than at other times of the year, maybe competitors were running a similar change in pricing or messaging which I can’t monitor but impacted customer behaviour. Either way, the test showed a real finding during the time it ran. But that finding didn’t apply to the always-on, messy and confusing real world.

The trouble with this testing is that it measures a moment in time, in certain conditions which we can’t fully control. We then try to apply those findings to the real world in different conditions, and the assumptions sometimes break.

To be clear, there are many cases where the test findings do hold out, and the real world matches our expectations. These tests are really valuable.

But they are not absolute truth, especially not when a single test is run in isolation.

What can you do to limit risk:

1. Replicate tests - can you replicate the results on multiple occasions, in different time periods, audiences and geos?

2. Validate tests - can you validate the performance on a different measurement method ? (e.g. MMM, surveys, attribution?)

3. Be skeptical - can someone spend a reasonable amount of time to find an alternative cause for uplift? (or lack of uplift?)

The problem with some of these mitigation methods is that they are time consuming. And they represent significant opportunity cost

I still love uplift tests, they can give really good insight into the real impact of marketing channels. But just because they sound quite scientific doesn’t mean they are always reliable.

Subscribe now

I have put together a comparison of 50 brands across 10 categories on the top 8 AI discovery platforms.

Each platform has a different audience base. The models and crawlers behind each model are also different, and frequent updates and launches make this a very changeable landscape.

So do they even agree on who is the best brand, who is the biggest brand and who to recommend the most?

This analysis looks at average rank for each brand within each category, and compares the results across 8 AI discovery platforms:

• Anthropic

• Deepseek

• Gemini

• Google AI Overviews

• Grok

• Meta

• OpenAI

• Perplexity

The brands themselves are across multiple categories (retail, technology, skincare, travel and food & drink).

We measure platform visibility on average rank, using tracking data through trakkr.ai

It is a consistent source, and uses the same definition of rank across all brands.

So what do we see?

Firstly do the different platforms agree on who to rank first in each category?

Not really…

The below shows the average rank in each platform for one brand in each category.

(Lower rank is better, a rank of 1 means that the platform is recommending your brand as the first recommendation, a rank of 3 means the platform recommends two other brands before recommending your brand. etc)

There is some general consistency, but also some big outliers.

All models agree that Brand 2 is big and ranks them highly. But even then there is some variation with Grok and Anthropic ranking them above 3, while all other platforms recommend between first and second position.

Brand 9 also gets strong overall ranking, showing at rank 2 or lower with 6 out of 8 platforms, but in this case OpenAI is the slight outlier pushing out to a rank of 3.

The lesser ranked brands show larger variation. Brand 3 ranks at 2.7 on Deepseek but a huge 9.2 on Grok.

Brand 8 ranks at 1.4 on Google AI summaries, but at 7.4 on Meta AI.

When a majority of value comes in being in the top three recommendations, that variation takes a brand from being in pole position to being totally irrelevant.

The range plot shows this a bit clearer. Some of the best ranked brands (Eg 2 and 9) have the lowest variation across models.

The lower ranked brands (eg 3, 5 and 6) have much more varied results.

But we still have strongly ranked brands, who get bad ranking on certain models (Eg 7 and 8) Average rank at around 3, but with one platform pushing them all the way out to above rank 7.

If we group up these findings across brands, then we can compare to the net average ranking across platforms.

This net average isn’t necessarily “correct” but it gives us a combined “platform crowd sourced average” to compare to. It helps us to see which platforms most often agree with each other, and which ones are bigger outliers.

DeepSeek, Perplexity and OpenAI show the most consistent agreement to this average. They appear to be the “most stable” rankings.

Grok has a completely different view on the world. It hardly ever agrees with the rankings from the other models.

Gemini and Google AI overviews show similar performance (As we would expect, because they share a lot of data sources). Anthropic and Meta are in the similar mid range, agreeing with the average around half the time.

If we look at how much they disagree with the average, we see some deeper nuance:

Groks average difference to mean is huge, as we would expect when it has a completely different ranking order.

Meta shows high variation. It only disagrees half the time, but when it does disagree, it disagrees a lot!

The others start to come in closer together. In general they agree quite a lot, and a lot of their disagreement are quite small.

So that is reassuring. Generally if we do the right things as a brand we can get a consistent improvement across 6 of the 8 platforms, including the 3 with biggest user uptake (Anthropic, Gemini and OpenAI).

But how consistent are those rankings by platform?

We can look at the visibility variation by platform. This looks at how much this average ranking changes within each platform from week to week and month to month.

We don’t want variation to be zero. Brand visibility will change over time. But we also don’t want it to change wildly week to week, which suggests we have no control over who is recommended and how.

Again we see a lot of different things in this chart:

Some brands show fairly consistent rankings across all platforms(eg Brand 1, 2 and 8). This is as expected, some brands won’t have made any movements in visibility.

Some show high variation across all platforms (eg 7 and 9). This is also as expected, some brands have gotten better or worse over time.

But some show consistent ranking on some platforms, while highly varied on other platforms. (eg 4, 6 and 10.) This is less expected, but seems to highlight the lack of consistency in what each platform values and how it is making decisions.

If we aggregate this variation up to a model level, then we can see average volatility by platform.

Lowest on Deepseek, Meta, OpenAi, Perplexity and Anthropic.

Highest on AI Overviews and Grok.

Again Grok comes out as the outlier in who it recommends, and how consistently (or inconsistently) it recommends them.

Surprisingly in this case Gemini and Google AI Overviews start to disagree, with ranking on Gemini being more consistent and less volatile than ranking on AI Overviews.

We see a similar breakout when we look at how often each model agrees with the crowd sourced average monthly change:

Grok and Meta (who already tended to not agree with the crowd sourced average) show weak agreement.

Deepseek, Anthropic and OpenAI quite consistently.

Finally, does this variation change if you are a big or a small brand?

It is tough to see, there is a lot of inconsistency in these comparisons. Sometimes being top ranked helps to make your ranking more stable, other times the top two positions appear to be less stable than the lower positions.

So, what have we learnt?

• If getting visibility on AI platforms wasn’t hard enough, it seems platforms can’t even agree amongst themselves on who to recommend. So we need to break down tactics by platform and match our priorities to who best covers our audience profiles.

• Ranking volatility is high, and inconsistent across platforms. We need to monitor several, not just one.

• Anthropic, Gemini and OpenAI have the biggest user uptake and the most consistent rankings and lowest volatility to other models.

• Gemini and Google AI overviews agree in most cases, but AI Overviews have higher volatility and less consistency.

• Grok has a mind of its own, and seemingly disagrees with everyone else a lot.

Subscribe now

The consistent insight we find across brands is that 10% of our attributed conversions from Paid Brand Search campaigns are driven by what we do in Paid Brand Search.

So the natural follow-on question is: what is driving the other 90%?

Let’s get into that one today.

What’s driving the other 90%?

As we went through last week, one of the huge benefits of the nested MMM answer to Brand Search is that we understand what is actually driving brand search volume.

People don’t search for brands by accident

We know that people don’t wake up in the morning and search for brand names for no reason.

Something must have happened to generate that search behaviour.

Knowing that your brand search demand is increasing is really valuable. It is a great indicator of long term success.

It tells you that you are doing the right things at a macro level.

𝗕𝘂𝘁 𝗸𝗻𝗼𝘄𝗶𝗻𝗴 𝘄𝗵𝗮𝘁 𝗶𝘀 𝗱𝗿𝗶𝘃𝗶𝗻𝗴 𝘁𝗵𝗮𝘁 𝗶𝗻𝗰𝗿𝗲𝗮𝘀𝗶𝗻𝗴 𝗱𝗲𝗺𝗮𝗻𝗱 𝘁𝗲𝗹𝗹𝘀 𝘆𝗼𝘂 𝘄𝗵𝗶𝗰𝗵 𝗼𝗳 𝘁𝗵𝗼𝘀𝗲 𝘁𝗵𝗶𝗻𝗴𝘀 𝘆𝗼𝘂 𝘀𝗵𝗼𝘂𝗹𝗱 𝗱𝗼 𝗺𝗼𝗿𝗲 𝗼𝗳.

This brand demand is made up of both long term slow changes and short sharp spikes.

You can see on the chart below, there is a long term base which is relatively unreactive to medium term marketing investments. Then there are some category seasonality impacts which create significant spikes, but we have little control over.

𝗕𝘂𝘁 𝘁𝗵𝗲𝗿𝗲 𝗶𝘀 𝗮𝗹𝘀𝗼 𝗮 𝗹𝗼𝘁 𝗼𝗳 𝗶𝗺𝗽𝗮𝗰𝘁 𝗳𝗿𝗼𝗺 𝗯𝗼𝘁𝗵 𝘀𝗵𝗼𝗿𝘁 𝘁𝗲𝗿𝗺 𝗮𝗻𝗱 𝗹𝗼𝗻𝗴𝗲𝗿 𝘁𝗲𝗿𝗺 𝗺𝗮𝗿𝗸𝗲𝘁𝗶𝗻𝗴 𝗶𝗻𝘃𝗲𝘀𝘁𝗺𝗲𝗻𝘁𝘀.

We run models which measure increase in brand search demand, alongside our models which measure increase in revenue.

It helps give a view to the longer term lens alongside shorter term performance.

So what makes up these contributions over time?

Every brand is different, and these behaviours will change for every brand in every market.

But when we compare across multiple brands and categories we see some trends.

• Around 15% 𝗼𝗳 𝘁𝗵𝗲 𝗯𝗿𝗮𝗻𝗱 𝘀𝗲𝗮𝗿𝗰𝗵 𝗽𝗿𝗲𝘀𝗲𝗻𝗰𝗲 𝗶𝘀 𝗱𝗲𝗽𝗲𝗻𝗱𝗮𝗻𝘁 𝗼𝗻 𝘁𝗵𝗶𝗻𝗴𝘀 𝘄𝗲 𝗱𝗼 𝘄𝗶𝘁𝗵𝗶𝗻 𝗣𝗮𝗶𝗱 𝗦𝗲𝗮𝗿𝗰𝗵. If you increase non brand search presence then some of those people will go on to search for your brand name. This effect tends to be bigger on competitor campaigns, some people in-market for competitors will discover you on competitor campaigns and come back on your brand terms (though the high relative cost of competitor campaigns can still make this a negative ROI impact.)

• Category seasonality usually represents around 25% of short term variation, but has virtually no impact on long term trends.

• Around half of brand search volume is driven by long term brand equity. The marketing activities you ran years ago. Most brands could turn off all paid marketing campaigns and keep around half your brand search volume in the short term. This obviously varies significantly based on brand size and duration, a long established brand like Coke Cola would see much higher brand equity impact, a newly launched brand would have virtually none.

.

The rest of the impact is from what we do in other marketing channels.

Brand Search is significantly reactive to TV / CTV. Driving both short term spikes and long term sustained growth (spot the dips when this investment is reduced / paused).

Even for ecommerce, digitally native brands, broad reach TV and CTV activity is one of the biggest levers to pull to move this brand search volume.

Paid social is a huge driver of search. We can see the really strong contribution from paid social engagement highlighted in red, and can see the significant impact it makes to the underlying volume across the brand before and after ramping up social engagement volume.

Some of the biggest sustained spikes in brand search volume can be attributed back to PR and influencer content. This shouldn’t surprise anyone. 80% of impact from influencer content is indirect (people not clicking on content, but showing up later in search or organic channels). These indirect uplifts are a crucial mid term growth lever for brand search volume.

Partnership activity is some of the hardest to measure, but brand search uplifts give a great short term proxy to show the longer term benefits. Highlighted below in yellow, these partnership activities (in this case, sponsorship of sports teams) drive both sustained long term impact and short term spikes through partnership amplified content. A brilliant mix of short and long term contribution while pushing the brand out to new audiences that would be hard to activate through paid marketing channels.

When we combine all those impacts together, we get 10% of our brand search volume is dependent on what we do in search in the short term, but only 4% in the longer term.

What we do in other paid channels is twice as important in the short term and nearly 8 times more important in the longer term.

Long term brand equity explains around a third of short term spikes and troughs, and more than half the long term trend.

Partnerships and Influencers give us some significant (but smaller) contributions in both the short and longer term.

And seasonality causes a lot of short term spikes but very little impact on the long term.

We know that brand search volume is a crucial measure of future success, but it has a lot of dependencies.

One of the weakest levers is what we do in search, especially in the longer term.

One of the strongest is how we show up in other marketing channels, particularly more broad reach channels such as CTV

We can make significant impacts through partnerships, PR and influencers. Which all tend to be undervalued by short term measurement.

Subscribe now

Across dozens of holdout tests and hundreds of MMM outputs, we consistently see the same pattern:

Paid Brand Search conversions are 10% incremental.

(technically 9.7% incremental, but let’s round up to make this article easier to read.)

That is a key finding from a number of models and tests we have run on paid brand search activity.

If you turn all the paid brand search ads off, then 90% of the purchases attributed to the paid search campaigns would still buy from you from your organic search listings.

It isn’t saying those campaigns do nothing, 10% of the attributed sales don’t buy from you on organic search. Those people made a search for your brand name, but because you no longer defended top paid search position they saw something else from a competitor and went to buy from them instead.

But 90% of the people who make a search for your brand name then go on to buy from you, will do so whether or not you have a paid brand search ad.

Two big questions

If that’s true, it raises two important questions:

1. How do we measure that 10% incrementality?

2. If brand search isn’t driving 90% of conversions, what is?

This article focuses on the first. We’ll tackle the second next week.

How do we measure incrementality of Paid Search campaigns?

Firstly, let’s start with the 10% number. Where does this come from?

Look in any Adwords account and you will find a paid brand search campaign which delivers large numbers of sales at a very strong CPA. These are serving ads to users who go to Google (other search engines are available, but really Google continues to dominate) and search for your specific brand name.

A lot of the people who make those searches click on your paid search ads, and loads of these people go on to buy products. Average click through rates can be as high as 30%, and conversion rates can be higher than 15%. Both several multiples higher than any other paid campaign would expect to see.

Those high CTR and CVRs net out to very strong CPA or ROAS values.

Clearly a lot of people who search for your brand, end up buying from your brand. Hardly surprising.

But there have always been long standing questions of how many of those people would have bought from you anyway if you turned the paid campaigns off.

They did search for your brand name in the first place after all.

There are a couple of ways of measuring that.

Method 1: Holdout Tests

Simply put, turn off the paid ads and see what happens to both paid brand search revenue and organic brand search revenue. If the paid ads are just canibalising organic then the sum of the two should be relatively unchanged.

Method 2: Marketing Mix Modelling (MMM)

In my view MMM should be an always on for any serious marketer,it is a vital measurement tool to triangulate towards an unknown truth. Run some models over time which look at the impact of changes in paid brand search spend and their impact on overall revenue. (Brand Search is a tricky element of MMM as the correlation will be high, but the causality is often low. You need to run a nested approach to these models, let me know if you are interested and I can write a different post which goes into a bit more depth on these nested structures, it gets quite complicated…)

I have run a lot of these tests over the years. Dozens of holdout tests, and hundreds of MMM outputs.

Two examples below:

Holdout test | During test period Paid Brand Search ads were paused. Post test Paid Brand Search ads were re-activated.

This graph shows index values of total revenue for paid brand search and organic brand search attributed revenue. All values are indexed to the pre test period average. (The real analysis was done on absolute numbers, but to maintain transparency these graphs share indexes.)

During the test revenue dropped by 5%. Post test, when campaigns were reactivated, the total revenue goes back to previous levels.

These graphs show indexes, but the absolute revenue volumes across these time periods were quite consistent. There were no other significant external changes during this timeframe.

When you split those revenue indexes by type it looks like this:

Obviously with Paid Brand Search campaigns paused that revenue drops to zero, but the SEO revenue increases from 38 to 95.

PPC revenue reduces from 62 to 0. But 57 of it (removing the 5% net reduction) is redirected to SEO brand revenue.

57/62 = 92%. So, in this uplift study, 92% of the Paid Brand Search Revenue is non incremental.

Gives us 8% Paid Brand Search incrementality.

Let’s take an example from the other method: MMM (Marketing Mix Models)

Measuring Brand Search in MMM is tricky. MMM runs through regression models, which are essentially time series correlation models.

And the correlation between Brand Search clicks (or spend) and conversions is high. So a simple, flat model will show a very strong impact of Paid Brand Search. (Often very strong. Showing the largest paid media channel with enormous ROAS numbers.)

But we know that a lot of this activity is navigational rather than causal (as our holdout tests show.) So we need to be able to identify the real driver of this activity, and separate out the causal, incremental, contribution of Paid Brand Search.

To do that you need to move away from those flat, simple models. So that we can allow the models to see the real drivers of demand rather than reflecting the correlation between searches and conversions.

There are 3 main ways to do that.

The 3 Approaches

1. Pretend Brand Search doesn’t exist

Remove it from the model, and make the model attribute those sales to the real thing which drove them. This is quite a blunt tool, but it does stop the over attribution to brand search. This is what a lot of self serve MMM SAAS toolkits recommend. It will help understand the contribution of indirect channels, but it doesn’t help us answer the question of paid brand search incrementality.

2. Embed priors into the model to reduce the impact of Paid Brand Search

This is the solution used by a lot of SAAS MMM providers. Bayesian modelling allows MMM to embed prior knowledge into the model build. So rather than the models just looking for a relationship in the data itself, they are also told “we expect this value to be approximately x”. The model then looks for a correlation number in the data, then tries to match it to the prior value. So if the prior says ROAS should be between 8 and 10, and the correlation suggests a ROAS of 30, then the prior will pull that ROAS down towards 10. This again helps to limit the impact of Paid Brand search and helps to identify other indirect drivers, but the model isn’t actually providing this answer, it is simply reflecting an answer which has been embedded into priors. 10 might be the correct answer, but this hasn’t actually come from the MMM output.

3. Build nested models which look at the drivers of brand search demand

This is the only way to really determine brand search incrementality through MMM. We have the initial MMM output (again let’s assume 30), and we build a secondary model which explains what is driving paid and organic brand search clicks. Those secondary models should show a long term baseline of brand search demand, generated from long term brand equity, then incremental demand generated from market seasonality, other marketing channels (for example brand search is highly reactive to TV and YouTube), PR, Influencers and a number of other factors. When you know how much of this demand is being generated from other areas you can reattribute this value back to those other channels and long term effects. What is left is the net impact of your paid brand search campaigns. This usually attributes up to 90% of brand search impact to other drivers, which would leave us with a net impact of ROAS =3 (30*10%). This method allows us to understand those indirect drivers, but also gives us the benefit of actually modelling the paid brand search impact. So rather than excluding (1) or hard coding (2) the paid brand search impact, we are actually measuring it.

So what does that look like in practice?

The below example runs through how we work paid brand search into these nested MMM structures.

This is for a high consideration, high price point purchase product. Average consideration time from lead to purchase is 6 weeks, and the target audience is restricted to the top 10% of earners.

When you run a basic flat MMM structure which includes brand search, then any regression model will show very strong contribution from paid brand search (particularly for these high consideration purchases.)

Paid Brand Search shows a 10% contribution to total sales. Even after MMM is accounting for category seasonality and other marketing channels.

Organic Search (of which 90% is on branded queries) shows a further 9% contribution.

For high consideration products, clearly a lot of people who make purchases do so with the last measurable interaction coming from branded search terms.

The raw data shows why these simple models will always highly value branded search terms:

The correlation between sales and brand search clicks is really strong. So any correlation model will show show very strong correlation between brand search clicks and sales.

But we know that the nature of brand searches are that people are looking for your brand.

It is an outcome of marketing as well as paid marketing channel.

So to understand the incremental impact of the actual paid campaign, we need to also understand “what is causing brand searches”?

We can do that through nested models. This gets quite complicated, but I have tried to simplify the language, remove the jargon and talk through in normal marketing language.

We run another regression model (alongside the sales model) which is trying to find the drivers for what makes someone search.

We need to be careful running those nested models, because several other factors will show strong correlation with brand searches (most notably any retargeting campaigns, or any on site behaviours) but will not be a driver of brand searches (as we know through the logic behind which action must have happened first).

To help isolate the potential drivers vs the actions which are correlated but not causal, we think about marketing activities as: User driven or Brand driven.

User driven actions: We can only engage with a user if they do something first. So we can only serve retargeting ads to a user who has already done the thing we want to retarget. Usually a site visit or engagement. We can’t double the audience we want to deliver to unless that audience does something first. In this case, we shouldn’t use these to describe secondary actions like brand searches, because the causality is likely in the other direction.

Brand driven actions: These are actions and engagements that we can control as a brand without requiring the user to engage with us first. Like paid social ads, or TV ads on awareness campaigns. If we want to reach twice as many people through paid social awareness campaigns then we can do that, we just have to spend more money. Twice as many people will see our ads without requiring them to do something first.

Then we have the elements which are not marketing levers, but are still impacts which are impacting how many people search for us. Things like market seasonality. More people will search for christmas trees at christmas. And that will happen whether or not we are running paid search campaigns.

So we also want to understand how much of our search is driven by external factors, like market seasonality, competitors activity, pricing, weather, news cycles. And loads of other things are very specific to individual brands (this is one of the reasons why you should always build these models specific to each brand, a one size fits all doesn’t really work in MMM.)

So we want to find the underlying causes of brand searches from actions which are brand driven, and what the impact is from external factors.

When we run those models we get outputs similar to the below. This is for a travel brand, a high consideration purchase with an average of 6 weeks from first engagement through to purchase.

Note. These type of models rely on having some variation in previous tactics and volumes. If you have always run at 100% impression share (or conversely never run branded search campaigns) then you will not get reliable results from this type of analysis.

We have the contribution from our paid brand search clicks in the simple model (reminder that this was at 10%)

But we also know that 90% of those brand search conversions is were actually from things outside of what we did in paid search campaigns.

52% of it is our long term brand equity. Turn off everything in marketing and that 52% of brand searches still exists.

12% is from paid social campaigns. These people are buying through search links, but only show up in those search listings because they found us on paid social.

Similarly 18% of the paid search actions are actually driven from CTVcampaigns

10% came from what we did in paid brand search campaigns. So if we turned off the paid brand search campaigns but kept all the other marketing activity running then we would expect to keep 90% of the brand paid search conversions.

This gives us our true incremental contribution for paid brand search activity at 10%. Very similar to the answer we got through the other holdout test method.

But validated through two different measurement approaches.

And measurement approaches which are truly independent of each other (there are some alternative ways of building MMM which hard or soft codes these other values into the outputs through priors)

So while the top level model suggested 10% of our total sales were reliant on paid brand search campaigns (a huge contribution, and by far the biggest marketing input), when we run the nested model structure we find that actually this is only 1% (10% of 10%)

Suddenly CTV and paid social are more responsible for the conversions which show up in branded search, than anything we do within branded search campaigns.

We run a lot of these models and tests. These results are very consistent across brands and categories.

We can see this across 20 brands of a wide variation of sizes and categories:

The volume of branded search conversions is highly varied, minimum at 4,152 conversions and a maximum at 43,313 conversions.

But all of them show significantly weaker modelled conversions after we run through the nested models.

Average incrementality is 9.68%

85% of brands see incrementality between 8% and 13%

There is only really one outlier at 23% incrementality, which happens to be the smallest brand in the sample set.

Choosing The Right Method:

Both methods give us a good view of the real impact of paid brand search campaigns.

Both also have pros and cons.

Turning off your campaigns has opportunity cost. If they are 10% incremental, then turning that campaign off cost you 10% of conversions over the test period.

And you need to turn it off for long enough to get statistical significance, which could be expensive.

You also need to restrict other changes during that test period. Which is both difficult and costly.

Running nested regression models is complicated, and requires past variation in the data. We have to have made changes in the past, but we can measure the past without creating more opportunity cost. There is also a greater depth of understanding needed to interpret these results compared to a simple on vs off test.

Having both options in your toolkit gives you the flexibility to use the right one for the right circumstances, and potentially use both to see if you get the same answer. If you get the same response from both methods (and keep them both independent of each other), then your confidence should rise significantly!

Subscribe now

Every campaign looks like it is successful when it runs at peak times. Meanwhile every campaign that runs out of season looks like it doesn’t work.

How do you separate these effects to know how good a campaign really is? Would the campaign in the offseason have worked well if it ran during peak?

If we can’t separate campaign effectiveness from market context, then we will never really know what is and isn’t working.

This is one of the key differences between measuring Media impact or measuring Marketing impact.

Why seasonality matters

So if accounting for seasonality is important, then how do we do that?

Typically there are two ways:

1. Remove the seasonality

2. Account for the seasonality

This sounds like it is the same thing, but there is a key difference.

Removing seasonality

Removing seasonality looks at past years of activity to see how higher or lower each month (or week, or day) than the yearly average. So if January tends to be 10% lower than the rest of the year, then we can increase the actual values for January by 10% to “remove the seasonality”. Vice versa, if November tends to be 25% higher than average, then we can increase the actual values for November by 25% to “remove the seasonality”.

Then when we run models on those de-seasonalised datasets, it is looking at the campaign impact after already accounting for the difference in seasonality in January and November.

Accounting for seasonality

Accounting for the seasonality is different. Instead of removing the seasonality from the data, we try to find the cause for it. If we are running analysis for a travel business, then we can find the number of people who booked flights on each day of the year. This tells us the underlying market seasonality (as well as some other demand drivers.)

When we run models using this method, we can keep the actual performance data, then include the market demand as one of the inputs into our models.

Why we prefer to account for seasonality

1. Separates the seasonal uplift into things we control and things we don’t control

We can’t control how many people booked flights in January (unless we are a massive market leader, but that is rare). But we can control our pricing, flight release dates and destination availability.

All of those things impact the previous yearly seasonality

If we have always run promotions in November, then some of the previous yearly increase of 25% is driven by our pricing strategy, not underlying seasonality. If we are removing the seasonality from the data then we are removing this pricing impact, accounting for the seasonality allows us to separate how much of the impact is from market demand and how much is from decisions that we can control.

2. Removing the seasonality stops us from learning how much better or worse a campaign works in peak vs non peak

Every campaign will perform better in peak periods, but that peak uplift isnt equal across all campaign types.

If we account for the seasonality then we can learn how campaigns would have performed without the seasonal support, but also learn how we should expect them to perform at each different seasonal period.

3. Allows us to adjust if the seasonality or market conditions change

Some seasonality is very consistent over time, retail peaks in November and travel peaks in Summer are easy to predict. But many seasonal demands are not consistent, fashion and health trends are inconsistent and unreliable. Applying previous yearly seasonal adjustments to the current year assumes that these trends are not changing, which is rarely a safe assumption.

Key takeaways

The big advantage to removing seasonality is data availability and ease of implementation. Adjusting for seasonality requires additional external datasets which are not always available, and often rely on assumptions of proxy data behaving how we expect. But if you can apply a strong proxy for demand, then it allows you to learn more and faster.

Understanding the impact of seasonality on your marketing performance is crucial, and using either of these methods is better than not adjusting for seasonality.

At Charlie Oscar, we tend to account for seasonality using tiered logarithmic models and by sourcing data which provides a reliable proxy for category demand. It gives the benefit of having model structure which aligns to how we strategically plan and execute media channels differently throughout the year.

Subscribe now