Overview
Machine learning-powered technology for creative direction
Services: Data science, Creative, Programmatic, Google Marketing Platform
Markets: UK
The challenge
We know that, when it comes to display advertising effectiveness, creative is the number-one factor in determining success. Our industry’s main technique for trying to make digital creative better is split-testing. But this approach is severely lacking, so Croud set out to build something better.
Working with online payments company Paysafe, we wanted to develop a solution that could isolate the visual traits of a display creative and reveal how each was contributing to performance. This would then enable us to make better informed decisions about key elements of campaign creative, including where to position the brand logo, which colours to use, and which calls-to-action to prioritise.
The Croud difference
Delivering the solution required a well-thought-out strategy, with several key steps:
- Using the Google Campaign Manager API, we retrieved all of Paysafe’s recent display creative, totalling over 2,000 ads.
- Working in Python, Croud used Google’s Cloud Vision API, the open source OpenCV library and custom-built scripts to recognise text and various visual features of the ads. We also trained bespoke visual recognition models using Google’s AutoML.
- For animated adverts, Croud used the Pillow, Selenium and FFmpeg libraries for Python, to break the ads into individual frames so that each frame could be analysed by the visual recognition software.
- Colour was determined by identifying dominant colours for each ad, and then grouping close gradients using scikit-learn.
- All the visual features of each ad and every frame were codified and logged in Google’s BigQuery data warehouse, along with the campaign configuration and performance data for every ad.
- To analyse all the data in BigQuery, we needed a model that could isolate the performance impact of all the individual features across the whole data set. We therefore built a Boosted Decision Tree model and used a Shapley Values approach.
- We displayed the results of the modelling in an interactive dashboard built using the open source Streamlit app framework.
Once the creative intelligence solution was ready, Croud chose a couple of creative formats that had been active on Paysafe campaigns in the UK. For those same campaigns, and a click-through rate KPI, Croud retrieved creative communications from the machine learning tool, and used these to create a modified version of the original creative.
The results
Running an initial test for two months, the Croud team found that the new, machine learning-led creative assets delivered a click-through rate (CTR) that was 131% and 89% higher year on year for skyscraper and banner ad formats respectively.
With this solution, the Croud team has developed an approach that not only enables us to judge fairly, because it controls for all the factors that contribute to performance, but also to optimise more effectively. As a result, we can realise bigger improvements, much more quickly.
The campaign was also recognised for Most Effective Use of Data for Creativity at the Drum Digital Advertising Awards 2020.
increase in banner CTR
%
increase in skyscraper CTR
%
confidence in results
This is very exciting. I’m looking forward to the impact this is going to have on our creative design and decision-making.
Annie Gregory, Content and Social Media Lead, Paysafe