Test, Test, Test

17 Feb 2025 » Opinion , MSA

If you have watched Mad Men, you will notice one detail about the marketing campaigns they created: it had to be a success right from the beginning, there was no room for errors. One of the reasons the main character (Donald Draper - Jon Hamm) is so successful, is because his campaigns are almost always perfect: they convince the client and be public alike. He could even convince the public to smoke more when health organizations were advocating for the opposite.

Digital Marketing

With the advent of digital marketing, there is no longer a need to be especially talented to create campaigns. Now we have thousands of people working in this industry, both from the business and the IT side. For example, my artistic skills are nowhere to be found. However, I can now use Adobe Firefly and create an image ready to be published.

Sure, campaigns will not be perfect the first time, but you can (and must) refine them through multiple iterations until you find the most appealing content to your customers. This is one of the reasons why I like digital marketing: almost anybody can do it.

And this is where testing comes into play. Everything I am going to explain below refers to the implementation phase, but you must start with the planning phase.

Web

If you are reading this, you probably have Adobe Target running on your website. Now, let me ask you: how many A/B tests are you running currently?

I do not have a benchmark that you can compare against, but here you have some experiences I have had with my customers in the past:

  • One customer was in the B2B business, selling corporate IT solutions. Since it was B2B, only some pages had enough traffic to run optimization campaigns, but they had the full year planned for those places where they could run such campaigns.
  • One of my closest coworkers approached me one day, towards the end of the year, asking for ideas. She was working with a well-known electronics manufacturer and they were in the planning phase. I asked whether it was for next year. To my surprise, she told me that next year was already planned and they could not fit any more campaigns. The ideas she wanted were for the following year, as the client was already asking.
  • An Adobe Target consultant showed me the results of an A/B test he had been conducting on a UK website. The goal was to optimize the checkout process. He could show an uplift of £1M (annualized) just by tweaking this checkout journey.

So, next time someone tells you that you do not need to run A/B tests or that just a few is enough, remind them that they are leaving a lot of money on the table. It does not matter if you have at.js or WebSDK, you can run Adobe Target on your website and increase conversion in many places.

Finally, if the test shows that a contender is better than the default experience, you need to make the winning contender the default experience. You can then stop the Adobe Target activity and everybody will get the best experience.

Emails

I believe that email content is, more often than not, overlooked when it comes to its optimization. I cannot think of any campaign I have been involved in, where the email content is tested. The idea is similar to web or app optimization:

  1. You create 2 or 3 variants of the same email. The change can be just the subject or the full body.
  2. For a small percentage of your population, you randomly send one of the variants.
  3. After a few days, you check which variant has received more clicks or driven most conversions.
  4. You send the winning variant to the rest of the audience.

Adobe Campaign offers this feature out of the box. Unfortunately, at the time of writing, Adobe Journey Optimizer (AJO) does not have this capability. However, you can manually create it by taking these steps:

  1. Create 2 or 3 variants of the same email.
  2. Configure your target audience and, based on it, 2 sub-audiences:
    • One will be a small sample of the target audience.
    • The other will be the rest of the audience.
  3. Create a journey for the sample audience, with a split activity and as many branches as variants. Each branch sends one of the variants.
  4. Run the journey and measure the results.
  5. Create a second journey for the rest of the target audience, using the winning variant.

I know, this is more work and I am wondering whether there could be a way to create it in a single journey. However, the result is the same as with Adobe Campaign: most of the audience receives the best experience.

Mobile

Now it is time to take the previous two sections and apply them to the mobile world. The reasons to do so are the same: optimizing the messaging leads to higher conversion.

In this case, you can optimize these aspects:

  • Content. Just as you do with the Web, you can run A/B tests on app content through Adobe Target.
  • SMS, push notifications and, in-app messaging. The same concepts I explained in the Email section above apply here: prepare multiple variants and evaluate which works best, making it the default message.

While SMS is simple, I am very well aware that marketing activities in mobile apps are difficult to manage. Adobe Target activities, and AJO in-app messages and push notifications often require the release of an updated version of the app. This is far from ideal because not all users will get the updated version.

However, this should not be an excuse not to optimize your mobile apps and messages. You should work with mobile development teams to make sure digital marketing requirements are part of the release cycle.

Digital Advertising

In one of the jobs I had before I got close to the Adobe world, I managed Google Adwords campaigns. Every Monday I would analyze the results of the previous week and try to optimize the campaigns I was running for my clients. There were two main levers I used:

  • Adjust the cost-per-click (CPC), to reduce it as much as possible.
  • Replace low-performing ads with new versions.

I have to admit that now I wonder whether I spent too much time and effort on these tasks, as the return I got was small. However, big corporations have significant advertising budgets and an increase in the conversion rate of 1% means a lot of money in savings or revenue.

Nowadays a lot of these tasks can be automated, but you still need some manual input. My recommendation is that you should not just let the software run your marketing; you should work on finding other ways to improve your digital advertising spend.

Destinations

So far we have talked about optimizing content (and, briefly, cost). However, there is more that you can optimize: the ad network or publishers. The idea is that your campaigns may not perform equally well depending on the medium used. For example, maybe Facebook brings lower quality traffic than digital newspapers because your potential customers read more news at source rather than through social networks. Be ready for surprises.

Adobe Audience Manager (AAM) has a feature exactly for this: Audience Lab. AAM creates mutually exclusive test segments, sends the audiences to various destinations, and measures the conversion broken down by destination. I know that AAM is not as sexy as it used to be, but if you still use it, take advantage of this feature.

Adobe Experience Platform (more specifically, RTCDP) does not offer a similar feature. You will have to do it manually, but it is still doable. To create the audiences, you can:

  • In real-time use cases, you will have to use segment builder. You would start by creating the base audience. Then, you would create as many audiences as destinations you want to test, starting from the base audience and adding a condition to create mutual exclusivity. This additional condition needs to be based on a random attribute. For example, if IDs are random, use the last character to determine which sub-audience a profile qualifies for. If you need more randomness, upon data ingestion you can calculate the hash of a concatenation of other profile attributes.
  • In batch use cases, you can build the audience through audience composition, by adding a split by percentage node.

You then send each mutually exclusive audience to a different RTCDP destination.

Measurement

Any test, whose results are not measured, is useless. In other words, you must establish a metric to evaluate the performance and measure it. The metric can be as simple as clicks, although ideally, it should be revenue. Therefore, you must make sure that you capture the conversion and its attribution.

Adobe offers some integrations to ease this process:

Things get more complicated with other tools. You will need to:

  1. Design a campaign ID structure to identify the campaign, the variant, the channel…
  2. Add a query string parameter with the campaign ID to all links pointing to your website.
  3. Capture the query string parameter.
  4. Send this parameter to good old Adobe Analytics or Customer Journey Analytics.
  5. Create reports to measure conversion based on each variant.

I may write a future post with a lengthier explanation of this process.

Still Thinking About It?

This post has become longer than I expected. However, if you have reached this point, are you still wondering whether you need to test all your digital marketing campaigns? I would argue that the question should be: can you afford not to improve your campaigns?

 

Photo by Kindel Media
Cultures icons created by Freepik - Flaticon

Related Posts