Meta explores the role of creative in marketing effectiveness

Data science consultancy, Ekimetrics, has teamed up with Meta to delve into the links between creative execution and marketing effectiveness.

Historically, measuring the efficiency of creative has been challenging, due to the difficulty in isolating its impact from other factors, such as execution tactics or brand health.

Ekimetrics used machine learning to develop a technical Marketing Mix Modelling (MMM) methodology that brands can use to understand which creative elements have the biggest impact on marketing effectiveness.

The study showed that the most efficient features across all sectors were ‘Product’ and ‘Person’; the two features were also shown to be important in combination.

For intangible products, such as insurance, attributes of a person, such as a smiling, visible face were key to driving the highest ROIs. However, each sector (and each brand) has variations in which elements deliver the greatest efficiency.

The machine learning-powered study analysed brands in four sectors – automotive, cosmetics, insurance and hospitality – encompassing almost 23,000 individual creatives from 2,300 campaigns across three and a half years, and with a total spend of $8.9m USD. “There is often a disconnect between brand and performance marketing, with brand playing second fiddle to other metrics” Matt Andrew, UK GM & Partner, Ekimetrics, said, “Developing our understanding of creative performance is the last frontier in marketing effectiveness. As brands add more creatives targeted to increasingly tightly defined audiences into their mix, making sense of what does and doesn’t work becomes an exponentially complex challenge.

We use data science in many ways in marketing, but the reality is that people engage most with the creative. And there is often a disconnect between brand and performance marketing, with brand playing second fiddle to other metrics.

“While you can test creative via third parties, the link between their metrics and sales performance isn’t strongly established. Nor has MMM typically evolved to capture creative well. This study and resultant methodology seek to bridge that gap. Good design and creative magic are of course essential ingredients in great advertising. But understanding the role of creative better will help brands to develop not just efficiencies in execution but in the creative process itself –whether across advertising or for specific applications such as product placement or sponsorship.”

Full results:

Brand Cue, Logo, Product and Person were the most common features that significantly drove ROI uplifts across brands.

Brand Cue was significant at driving higher ROI in Hospitality and Insurance. It could, however, not be tested for Cosmetics and Automotive because of low impression levels (3% and 0% respectively)

Logo was significant in Hospitality and Cosmetics.

For Automotive, 74% of creative impressions featured a Logo (either on the car, above text, or in the corner of the creative), making it difficult to separate the effect of Logo from other features appearing in the creatives. Therefore, Logo did not achieve the expected significance.

The features with the largest uplift across brands (ranked 1) were Product, Text and Person with Face; Text was the most efficient in Cosmetics, Person with Face in Hospitality and Automotive and Product in Insurance.

The combination of Person & Product ranked second in both Cosmetics (with Face) and Automotive (without Face). Brand Cue was also a significant feature in the analysis, ranking sixth and seventh for Insurance and Hospitality, respectively.

Similarly, Logo ranked sixth for Hospitality and Cosmetics. When combined, these two features were found to be even more efficient for Insurance and Hospitality, as Logo & Brand Cue was ranked second for Insurance and fifth for Hospitality.

For Hospitality, Medium and Small Logo were ranked second and fourth, respectively, while for Cosmetics, Medium and Small Logo were ranked fourth and fifth respectively. Both results indicate that having a bigger Logo is more efficient than having a Small Logo, and in general, having a Logo is better than having no Logo at all.

The scale of the study demanded the use of AI and machine learning, with 124 different models in play across multi-stage econometric modelling, and object and text detection (for example, logos, brand cues, products and copy).

Critical to the methodology was the use of both pre-trained and custom-trained models for optimal object and text detection performance, with the model selection itself requiring an understanding of the relative merits of when and which pre-trained models to use and how and when to create custom models.

The methodology can help provide brands with significant insight into creative development and execution. While the results aren’t a blueprint for brands for which creative elements to include, the methodology can help find a particular brand’s sweet spot.

The brands involved granted permission to Ekimetrics to use Meta data, accessed via Ekimetrics’ partnership with Meta.