How do you test ad creative effectively on Facebook and Search?
If you’re reading this, you’ve either invested considerable resources in building new ad creative or you’re planning to.
How do you make sure the new ad creatives are being tested effectively on each channel? That’s what this guide is for.
Overall Ad Testing
The two biggest ad platforms, Facebook and Adwords, each have their own nuances, which we’ll dig into below.
But, first, here’s two things that are the same for all modern ad platforms.
RULE 1: Ad platforms use algorithms to allocate spending to ads,
and you shouldn’t try to force volume.
a. 20% of your ads will get 80% of your ad spend. This is because ad platforms assign spend quickly to the ads they think will get the best performance.
b. Unless you have knowledge that the platforms don’t have (for instance, you can see down funnel conversion data but the platforms can’t) you should not manually interfere by forcing the platforms to serve ads evenly. It’s almost always a mistake to force ad spend on ads for the sake of learning.*
c. Thus, the best performing ads can’t be measured by purely looking at cost per conversion. You have to look first at how much spent the platform assigned an ad in the time period, and only compare it to ads with similar amounts of spend.
In the below example, are ads 2 and 3 better than ad 1 since cost per result is lower? Absolutely not, if you try and equalize spend ad 1 will win every time.
d. Best use of the ad platforms means taking advantage of their algorithms - that is, letting the system pick where and how to target your ads whenever possible.
*Why is it a mistake to force ad spend on ads for the sake of learning?
Some people think the trade-off of learning something is worth the hit to the ad algorithm that comes from forcing spend. Here’s why that’s rarely the case, using Facebook as an example.
Say you want to learn a generalized factoid about your ad performance - like, “Do ads with cartoon images perform better than ads with stock photos.” The hypothesis being that if you learn which one works better, you can design better ads in the future by focusing on those.
In order to learn this with certainty, you’ll need a fairly sizable test. When ad platforms optimize ads in the background, they use a lot of black box estimation techniques to get results faster. We don’t have that luxury, so depending on the effect size we’re looking for, we’ll need 50-100 conversions on each side of the test to get stat sig data.
In many B2B accounts this is would be a sizable portion of total spend, so you have to figure out how to run this spend without overly disrupting your main campaigns. If you do this within your main prospecting campaign, you’ll have to disrupt the algorithm to change campaign settings. If you do it in a new campaign, it can cannibalize from your main prospecting efforts.
Even after you get this learning, it depreciates very quickly. Say that stock photos wins and perform 15% better in the test. You switch all new production to stock photo ads. Users then fatigue from seeing your new style, and cartoon style ads start winning again.
In summary: It’s best not to test at too high of a level. Just try to make ads that beat existing ads.
The one exception is if you want to test something fundamental for your brand, like a new logo - then the learnings can be useful enough across the board for your business to make the hit to your ad performance worth it.
RULE II. Ad testing requires conversion volume, which you don’t have a lot of.
If you’re running a campaign to get people to download a gaming app, your cost per conversion could be as low as a few $. Testing creative is fast and easy.
In B2B, the cost per conversion is anywhere from $100 - $1000. You have much fewer conversions to learn with.
What does this mean?
a. Until you have 30 conversions per week per ad-set, take generalized learnings with a huge grain of salt. Rely on the ad algorithms to pick ad winners for you.
b. To get more data to test ads with, consolidate your account into fewer campaigns where possible. This means your campaigns can learn to target and promote the best creative much faster. Example below:
Below, we’ll get into the specifics of each ad channel.
On Facebook, there are two approaches that work for testing the new ad creative.
Option 1: Creative Reinforcements - This means just reinforcing your ad-set by pausing low performers and adding fresh creative into your ad-set directly on a weekly basis.
a.Use Case: Provide the Facebook algorithm with new creative on a regular basis to compete with your incumbent creative
b. Approach:
i. Pause your low performers and add fresh creative into your ad-set directly on a weekly basis.
ii. Low performers include ads which have gotten little spend
c. Additional Considerations:
i. It can be difficult (and sometimes frustrating) to unseat the incumbent creative. Often times the new creative will receive little spending and Facebook will continue to spend against long term creative
ii. This is to be expected as Facebook is good at determining the strength of new creative with very little spend
iii. If this continues after numerous creative cycles, it is worth using Option 2 to build stronger challenger creative to go up against the incumbent
Option 2: Dynamic Graduation - a new approach RP has been using that’s been working well.
a. Use Case: Test ad copy and creative combinations at scale to determine the best combination to use for your evergreen campaigns
b. Approach:
i. Have two ad-sets within a CBO campaign.
ii. One ad-set is evergreen and contains your strongest performing ads.
iii. Another is a new Dynamic ad-set, placed each week or two.
The Dynamic ad set tests up to 10 new image concepts and up to 15 new copy concepts (5 primary text, 5 headlines, 5 descriptions) in Facebook’s dynamic ad format
a. The winner of the dynamic ad-set gets placed in the evergreen ad-set each week.
b. This dynamic ad-set is then paused, and a new dynamic ad-set with 10 more images is placed the next week.
AdWords
Google will sunset Expanded Text Ads (ETAs) on June 30, 2022. At this point, advertisers will no longer be able to create or edit ETAs. This means all ad testing in Google should be done via Responsive Search Ads (RSAs) given their increased importance in the Google Ads infrastructure.
Responsive Search Ads allow an advertiser to upload up to 15 headlines and 4 descriptions (pinned or unpinned). Google takes this portfolio of assets and creates the strongest ad for each auction based on the campaign goal, historical asset performance, and user signals. There is limited advertiser control and feedback data associated with RSAs, impacting what is possible from a testing perspective. Based on your end goals, there are three main options for testing ads on Google.
Option 1: Consistent cycling of RSA assets
a. Use Case: Incrementally improve the strength of your ads over time by providing fresh ad copy to go head-to-head with proven winners
b. Approach: Pause down the 5 lowest-performing headlines and 2 lowest performing descriptions and replace them with new assets
i. Google shares limited asset-level data. Performance can be listed as Best, Good, or Low but often times assets don’t have a rating or all assets have the same rating.
ii. Because of this, impression volume should be used as the proxy for performance. You can view asset level impression volume at a campaign level if you want to understand aggregate performance across similar campaigns
c. Additional Considerations:
i. It can be hard to unseat a proven winner from an RSA with lots of history. If cycling RSA assets isn’t producing new winning assets, it can be worthwhile to launch a brand new RSA with both new and old assets. This can give the original and new assets a more even playfield when competing for auctions.
ii. Google will show the top combinations of assets for each ad. This can be useful when brainstorming new ad copy and understanding how and where Google is placing assets
Option 2: Use the Ad Variation Tool in Google Ads
a. Use Case: Understand how significant additions/edits in ad copy messaging impacts performance. There are limits to what can be tested (due to tool limitations) but test examples include:
i. How does referring to your product as a Digital Asset Management Platform vs. Creative Ops System impact performance?
ii. How does adding Headlines with pricing impact performance?
b. Approach: Set up a new Ad Variation in the Google Ads platform
i. Can be set up at the account or campaign level, or across multiple campaigns.
ii. Can filter the ads you plan to include in the Ad Variation.
iii. Can find and replace, add or remove assets
iv. Can set the user split and duration for the experiment
c. Additional Considerations:
i. Some of the functionality in the Ad Variation Tool is no different than what is possible when cycling assets in an RSA (i.e. add 3 new headlines). The difference is the Ad Variation Tool allows you to have control and therefore a clean readout on the impact of adding/removing assets
Option 3: Create a Custom Experiment at the campaign level
a. Use Case: Understand how completely different approaches to ad copy impacts performance. This approach allows you to run unique ads on each side of a campaign split.
b. Approach: Approach: Set up a new Custom Experiment in the Google Ads platform
i. Select your base campaign. Recommend condensing live ads to 1 RSA per ad group to ensure you are able to compare ad vs. ad across sides of the experiment
ii. Update the ads on the variant side with the test ad copy
c. Additional Considerations:
i. This approach will potentially be the most disruptive to your account and requires the most backend setup. Because of this, it should only be used to test significant changes in your ad copy.
ii. Similar to the Ad Variation Tool, you are able to set the traffic split at launch for Custom Experiments. This can hedge the risk of pushing a high volume of spend against unproven ad copy