Creative Testing on Meta - Methodology explained

I’m sure you’re already familiar with the fact that 70% of a campaign’s success depends on creative. That’s why you test your creatives.

The importance of creative testing is nonnegotiable, yet execution comes with challenges. 

Modern marketers often struggle with getting enough creatives to test, deciding what to test, and asking the right questions to get the answers through testing.

But most importantly, the biggest challenge comes from how do we perceive creative testing. 

This isn’t just a handy tactic for performance marketers, it’s an investment in smart business decisions. Creative testing provides answers about which kinds of ads resonate with your audience and where to invest your ad budget, ultimately improving your ROAS.

When executed as part of a marketing strategy, creative testing enables you to continuously improve your ads, giving you a competitive advantage. By re-analyzing winning approaches and producing better creatives each round, you'll stay ahead of the competition.

So, my focus for this article is to explain how to execute creative testing on Meta and to understand that it is not only about testing blue vs. red.

How should you approach creative testing (on Meta)? 

Though it is called ‘creative’ testing, it requires a completely scientific approach, to avoid bias or prejudice. The methodology is what makes the difference, and usually, this is exactly where most marketers make mistakes. That’s why the following approach is not only to be applied on Meta but in testing in general. 

These are the four steps you should not skip:

1. Define the business goal.
2. Generate a hypothesis to be tested.
3. Design a test to verify the hypothesis.
4. Plan the next steps based on possible outcomes before the test goes live.

I’m going to describe this approach through the example of an eCommerce fashion brand:

• Business Goal: Determine which type of ad creative drives more sales for a clothing line.
• Hypothesis: User-generated content (UGC) featuring real customers will outperform professional studio photos in terms of conversion rate.
• Test Design:
  • A/B Test: Compare the performance of UGC ads against professional studio photo ads.
  • Metrics to Observe: Conversion rate, click-through rate (CTR), and return on ad spend (ROAS).
• Next Steps Based on Outcomes:
  • If UGC wins, use UGC in the main campaign.
  • If professional photos win, use professional photos in the main campaign.
  • If there is no clear winner, use both types of creatives in the main campaign or potentially reevaluate the test design.

How to properly design a test?  

The business goal and hypothesis are straightforward while designing a test needs further explanation. So let’s dive into it. 

The best case scenario when executing creative testing is to design a test that will represent your usual environment. When designing a test, you must choose between increasing the reliability of the results or reducing the cost and time needed to run the test.

When balancing cost and reliability, you need to make choices about how big and detailed the test will be. These choices can make your test more accurate or save you time and money. Here are some examples:

• Running ads for a shorter period will save time but might not give enough data to show which ad performs best
• Testing with a smaller audience will save money but might not represent your entire customer base.

It's important to consider the costs and benefits of your choices because they affect how reliable and scalable your test results will be.

Here are general guidelines when designing a test: 

• Ensure consistency by isolating testing variables
• Use consistent KPIs across tests for fair comparisons
• Apply the same budget, bid, targeting, and optimization for each test group
• Avoid edits during the test; if necessary, apply them equally to all groups

Start off with the biggest questions you want answers to and keep it simple. Do two to three different ads and build upon that. Once you get in a rhythm of it, you'll realize that it becomes a ritual.
‍
Grace Robertson
Head of Creative at IN Digital

Types of creative tests you can run on Meta 

When discussing creative testing on Meta, the first association is a split test, more precisely the A/B test. But this is just the tip of the iceberg. Split tests are a part of standard tests, that are faster and less complex to run. 

Apart from the standard ones, there are also advanced tests, more complex in both design and setup, but with more detailed insights into the creative performance. The typical example of this creative test on Meta is a Conversion Lift Study, which brings incremental results to your campaign.

Ad Ranking

This type of testing relies on Meta’s auction algorithm, that optimize delivery towards the best-performing creatives. Advertisers use this type of creative test to choose top-performing ads.

However, relying solely on ad ranking for creative testing on Meta can be tricky. The auction algorithm is biased towards early performance, not giving all creative candidates an equal chance. This can result in misleading results. While this method is cheap to run, it lacks reliability and statistical significance, which is why the best way to use it is as a part of the testing framework.

Split test (A/B tests) 

This is by far the most popular and often performed type of creative test. A/B testing compares two versions of an ad by changing variables like images, text, audience, or placement. Each version is shown to different segments of your audience to determine which performs best.

A/B testing is usually easy to run, but what really makes the difference is your setup. First of all, you must choose the success metric, most relevant to your campaign objective. Also, it is very important to pick the variable you want to test.

It’s not only launching a basic split test of if purple or red works better. It’s rather about understanding your brand and your position, so you can start to tailor your messages according to that.
‍
Gulce Ozman

The advantage of this test is its fast and simple execution and the fact that can easily be incorporated into a wider testing framework.

Holdout test (Conversion Lift Study) 

Holdout tests are advanced tests, that have a similar mechanism as split tests since in both ways you’re splitting your audience into two groups. The notable difference is what you are showing, or not showing to them, and what kind of results you’re getting.

Holdout or lift tests measure the real impact your campaigns have on the revenue, giving you incremental results, by comparing groups that had the opportunity to see your ads with the group that hadn’t seen them (holdout group).

The most recommended holdout test on Meta is the Conversion Lift Study. 

This test works as explained above: Meta randomly splits your target audience into two groups, where the test group sees your ads, and the control group doesn’t. Most tests take 3 to 4 weeks to complete.

The main advantage is robust, detailed results. The downside is they require more time, budget, and a complex setup. That’s why usually advertisers need help from their Meta rep, or platforms like Hunch to really take the most out of such testing.

Real-life example Conversion Lift Study 

To illustrate my point, here’s an example of a Conversion Lift Study we did for a customer. 

Meta's marketing science team ran a 4-week Conversion Lift Study to measure the increase in ROAS. Using Hunch's AI-powered tools, they removed backgrounds from 350,000 product images to create enriched DPAs.

The test results showed that Hunch outperformed both competitor overlay and no-overlay templates, driving a 2.3x increase in ROAS. Partnering with Hunch is estimated to save you 64% in annual costs.

Creative Testing on Meta - DOs and DON’Ts 

I’ve gathered our customer experiences to create a simple and handy list of creatives testing DOs and DON’Ts.

Conclusion 

Here’s what I want you to remember after reading this article. 

Creative testing shouldn’t be a one-hit wonder or a quick fix when the budget shrinks and you need fast results. Instead, creative testing should be your ongoing success strategy, continuously improving your advertising output. I know this is easier said than done with all the challenges modern marketers face, which is why Hunch is here to offer full support in both the creative and strategic aspects of creative testing on Meta.