Measuring Impact: The Beginner’s Guide to Incrementality

Do you know the impact your marketing campaigns have on your business? Traditional metrics only scratch the surface of this question. Incrementality solves for this by going a few levels deeper.

Think of incrementality as the North Star of marketing. By measuring for it, you can begin to understand the positive, negative, or neutral impact an ad has on your business. Industry experts have taken notice and begun to base their media-buying solely on incremental gains.

What Is Incrementality?

While an increasing share of the industry is looking to move away from last-touch attribution, the majority of marketers still depend on it due to its ease-of-use. While this is a simple way to assign credit, it’s not the most accurate.

Assigning credit to the last marketing interaction is the equivalent of going to the movies and giving 100% of the credit to the poster outside the theatre. Even if you had already seen a TV commercial and a newspaper ad, heard a radio spot and been served display ads, a last-touch model wouldn’t give credit to any of these touchpoints. In spite of not capturing the entire customer journey, this method is often used by default due to its simplicity. However, simple doesn’t mean accurate.

Incrementality isn’t about assigning credit to a conversion; it’s about identifying the interaction that moves a user from passive to active. Whichever interaction influences an actual outcome is identified as incremental.

A Closer Look Into Incrementality

Incrementality is a way to measure an event that wouldn’t have occurred without a specific interaction, such as an ad view, and that resulted in the desired outcome, such as a conversion.

In the context of ads, it can be referred to as the measurement of ad effectiveness.

Here are some key questions incrementality can help answer:

1. 1. What happens if I stop spending with Vendor X today?
   2. What happens if I increase or decrease budgets for Channel X?
   3. Which ad creative is the key contributor to my wanted outcome?

These questions can be summarized by asking: Do my ads drive actual value, or just claim credit for an action that would naturally occur?

Incrementality strives to identify the causal event of a conversion, allowing businesses to properly allocate budget and reduce wasted ad spend.

What Incrementality Isn’t

Incrementality and lift have become buzzwords in the ad industry. To understand what incrementality is, it’s important to understand what it isn’t.

Many vendors have positioned incrementality as: Vendor X drives 10x return on ad spend (ROAS). We can drive 15x ROAS—this means we can deliver a 5x lift. Unfortunately, measuring it isn’t that simple.

A 10x vs. 15x ROAS could indicate that one vendor is better at crediting conversions. Generally, this type of “lift” is unrelated to the actual ad treatment and indicates that one optimization engine is better at finding users who are likely to convert. In this case, rather than adding more conversions, the vendor simply takes more credit out of the existing total converting pool.

It is often unlikely that the claimed ROAS is reflected in the overall revenue baseline. In reality, it may only drive a portion of the credit claimed; or in the worst case scenario: there is no impact at all.

How Does Incrementality Differ From Traditional Optimization Methods?

The emergence of vendor accountability

The notion of incrementality has already increased vendor accountability. Accountability measures include monitoring fraudulent behavior, measuring viewability, using ad blockers, and observing brand safety guidelines.

All of these methods work toward a common goal of holding both vendors and publishers accountable for delivering actual results, thereby improving the user experience. Unfortunately, goals such as driving a low cost per acquisition (CPA) or improving click-through rates (CTR) are in direct conflict with truly impactful outcomes. It’s important to know how these goals influence the way vendors select and develop their bidding systems.

Click optimization (CTR)

Click-optimization algorithms try to accomplish good click performance by looking for users most likely to click, and heavily clicked placements (e.g., mobile). However, studies have shown not only that few people click on ads (Fulgoni & Morn 2008) but also that the people who click don’t correlate with actual converters (Dalessandro et al. 2012a). According to Nielsen and Facebook’s 2016 study on brand effect and CTR, clicky users are often more than 5.5 times more costly than non-clicky users, but reaching them does not impact overall sales. This is because fewer people click on ads, but multiple vendors compete to win credit for each click.

Click optimization, although profitable for vendors and useful for scaling advertising spend, aggressively targets only a subset of users. This not only creates a bad user experience but is also prone to click fraud.

Conversion optimization (CPA)

Rather than pursuing the ad click of any user, conversion optimization looks for people who are likely to convert. To increase the likelihood of receiving credit for the conversion, the channel delivering the ad must focus on two factors:

1. Recency: People who recently visited the page or are still on the page.
2. Intent to purchase: People who show a strong intention to convert (e.g., cart abandoners).

These two factors indicate that someone is likely to convert by default, regardless of ads. The engine then enforces heavy ad treatment when the user is close to converting. The benefit of this model is the high scalability for the ad vendor since it doesn’t look for driving an outcome but instead claiming credit for an outcome that is very likely to happen.

Last-click conversion optimization (cCPA)

For many years, marketers have measured performance by last-click conversions, relying on the assumption that a click “clearly” indicates a direct link to a conversion. To be successful with last-click conversions, vendors focus on combining click and conversion optimizations. This results in heavy ad treatment, typically right before a conversion, and has been identified as a key driver of negative lift: a portion of visitors who were going to convert won’t convert due to ad overexposure. This, combined with the risk to brand reputation, can be very harmful to businesses.

Incrementality optimization (iCPA and iROAS)

Incrementality looks for the causal event that drives the desired outcome. In the case of ads, this means identifying the positive, negative, or neutral impact an ad has on a business outcome. Accurately identifying causality is a long-standing problem in data science and isn’t as simple as click or conversion optimization.

Naturally, most marketers would only want to spend money on channels that drive results and ensure a positive brand experience. With data-driven attribution measurement platforms rapidly growing, measuring impact is becoming more and more accessible. That being said, the ad industry does not optimize for users who convert due to ad treatment only. We can see a movement toward incrementality across the industry, mainly in the realm of measurement, but the actual optimization is complicated and can be seen as conflicting with individual vendor goals.

The perceived conflict with individual vendor goals is best explained by an example. Let’s say an advertising platform needs incrementality to prove the value of its display and video channels, but incrementality is in direct conflict with its search product—one of its core business units. Search works based on a click model that is highly profitable due to its scale, but it is unclear whether search or branded search produces actual incremental results. Incrementality optimization, therefore, becomes a risky business model. By reporting on incrementality, the integrity of click-centric search products, which drives a large share of revenue, could erode.

The question on the vendor’s side is whether or not incrementality is scalable and profitable for the business. Doing the right thing for marketers and end users could, therefore, be seen as a potential conflict with business profit. This conflict makes incrementality optimization a high-risk investment for vendors.

Even with the hesitancy from some vendors to evolve into incrementality, there is a shift towards accountability, making incrementality optimization unavoidable. Once the market is more educated, the pressure will be on vendors to measure and sell their media incrementally.

Last updated on September 20th, 2023.