Author

Siddarth Mohan

Director Data Science & Global Lead for Causal AI Research & Marketing Mix Modelling at Artefact

The Bridge - Data Coffee

Emmanuel Malherbe, Head of Research Center at Artefact, met recently with Sid Mohan, Director Data Science & Global Lead for Causal AI Research & Marketing Mix Modelling at Artefact, to talk about marketing measurement.

There are many moving elements in the field of marketing measurement. Could you highlight what innovations will fuel marketing measurement in the future?

In our measurement framework, we have three methodologies: attribution, incremental testing, and MMM. However, these methods currently face constraints preventing seamless collaboration.

The attribution model, for example, needs to include the incorporation of constraints related to the baseline, a critical factor in understanding the overall impact of marketing activities.

Simultaneously, incremental testing is often tightly controlled by major tech platforms. They often require significant media spend for execution and are often limited to paid digital media alone. This necessitates advanced techniques like Bayesian causal inference and the potential outcomes framework; methodologies that have been pioneered and open-sourced by entities like Amazon Science and Microsoft Research.

And in the area of MMMs, there are solid opportunities to leverage the latest advances in causal models, again thanks to the pioneering work of Amazon Science, whose breakthrough innovation is indeed open source and available for the data science community to build upon.

So, how do these advanced methodologies address the challenges faced by traditional models?

The current methodologies offer a more nuanced understanding of marketing activities within the MMM framework. However, they resist going into the granularity necessary for marketers to take swift and targeted actions aligned with specific business needs. To address this, there’s a surge in the development of hierarchical Bayesian models and more importantly, structured causal models, which allow a more granular perspective and more frequent execution of MMMs.

Moving to the topic of causality, there seems to be a shift towards understanding the “why” behind data. Can you tell us more about this?

Since 2021, there’s been a causal AI revolution. Companies, led by Microsoft Research and Amazon Science, are now emphasizing understanding the “why” behind data, transcending the traditional focus on the “what.” Open source solutions in structured causal models enable businesses to estimate the impact of a single event within the broader context of concurrent and confounding events.

Fostering a culture of innovation within a company is crucial. How do you advise companies to create and inspire experimentation and innovation?

  • Have the courage to start asking the questions that matter. Evolve your learning agenda to identify the questions that current measurement methodologies are incapable of providing answers to.

  • Collaborate with data science teams to define and hypothesize expected benefits and value add-ons obtained from answering those critical questions. For example, is it more beneficial to have insight into which creative formats deliver the best returns or which geos should be the focus of my lower funnel campaigns?

  • Test new methodologies in a controlled environment and compare over-arching results with existing solutions to ensure business logic. For example, if you have a more granular MMM based on causal models, do the results remain in line with a classical channel-level MMM that is already in use or are the results completely contradictory?

  • Be comfortable with failure. The pace of developments around Marketing Measurement is accelerating just as in the rest of the AI space. Not every new approach or model is going to generate value. But that shouldn’t discourage you from keeping up-to-date with trends in the AI community and always creating space for experimentation and testing.

  • Instill an innovation mindset within both marketing and data teams. The biggest hindrance to building a culture of innovation and experimentation is the institutionalization of things. When the phrase “that’s always the way we’ve done it” is mentioned, it means you’re stagnating. Look at some of the top companies like Booking.com, Uber or Spotify who constantly stay in tune with what their customers want; it’s because they continuously update their offers and services. They constantly measure, as close to expected reality as possible, and no matter how hard it might be, certain campaigns or marketing efforts. Imagine, for example, trying to measure the true incrementality of Spotify’s Unwrapped 2023, one of this year’s most trending marketing campaigns.

The synergy between marketing and data teams is crucial. Can you elaborate on how these teams can effectively work together to drive innovation?

Marketing and data teams are an ideal pairing, with marketers driven by a desire to push creative boundaries finding a match in data teams that thrive on solving challenges. Breaking away from institutionalized approaches is crucial for fostering an innovative culture. Encouraging data scientists to solve problems that arise from pushing creative boundaries ensures the company remains agile and relevant.

Can you share an example where a company successfully embraced this culture of innovation?

In the Netherlands, we’re working with an online e-commerce player that conducts hundreds, if not thousands of experiments annually, a major shift from having no measurement practices just two years ago. This robust culture of innovation and experimentation has set them apart, improving marketing efficiency and ROI by an impressive 20%.

In summary, the synergy between advanced measurement practices, a focus on causality, and a culture of innovation and experimentation is reshaping the marketing analytics landscape and ensuring that companies remain agile and relevant in an ever-evolving business landscape.

Visit thebridge.artefact.com, the media platform that democratizes data & AI knowledge, in videos & podcasts.