We know that, when it comes to display advertising effectiveness, creative is the number-one factor in determining success. Our industry’s main technique for trying to make digital creative better is split-testing. But this approach is severely lacking, so Croud set out to build something better.
Working with online payments company Paysafe, we wanted to develop a solution that could isolate the visual traits of a display creative and reveal how each was contributing to performance. This would then enable us to make better informed decisions about key elements of campaign creative, including where to position the brand logo, which colors to use, and which calls-to-action to prioritize.
The Croud difference
Delivering the solution required a well-thought-out strategy, with several key steps:
- Using the Google Campaign Manager API, we retrieved all of Paysafe’s recent display creative, totaling over 2,000 ads.
- Working in Python, Croud used Google’s Cloud Vision API, the open source OpenCV library and custom-built scripts to recognize text and various visual features of the ads. We also trained tailored visual recognition models using Google’s AutoML.
- For animated adverts, Croud used the Pillow, Selenium and FFmpeg libraries for Python, to break the ads into individual frames so that each frame could be analyzed by the visual recognition software.
- Color was determined by identifying dominant colors for each ad, and then grouping close gradients using scikit-learn.
- All the visual features of each ad and every frame were codified and logged in Google’s BigQuery data warehouse, along with the campaign configuration and performance data for every ad.
- To analyze all the data in BigQuery, we needed a model that could isolate the performance impact of all the individual features across the whole data set. We therefore built a Boosted Decision Tree model and used a Shapley Values approach.
- We displayed the results of the modeling in an interactive dashboard built using the open source Streamlit app framework.
Once the creative intelligence solution was ready, Croud chose a couple of creative formats that had been active on Paysafe campaigns in the UK. For those same campaigns, and a click-through rate KPI, Croud retrieved creative communications from the machine learning tool, and used these to create a modified version of the original creative.
Running an initial test for two months, the Croud team found that the new, machine learning-led creative assets delivered a click-through rate (CTR) that was 131% and 89% higher year-over-year for skyscraper and banner ad formats respectively.
With this solution, the Croud team has developed an approach that not only enables us to judge fairly, because it controls for all the factors that contribute to performance, but also to optimize more effectively. As a result, we can realize bigger improvements, much more quickly.
The campaign was also recognized for Most Effective Use of Data for Creativity at the Drum Digital Advertising Awards 2020.
increase in banner CTR
increase in skyscraper CTR