A/B Testing for Email Campaigns
A/B Testing for Email Campaigns
A/B testing allows you to send two variations of a campaign (A and B) to a subset of your audience, evaluate which performs better based on your chosen metric, and automatically send the winning variation to the rest of your subscribers. This feature helps you optimize subject lines and email content to maximize your open and click rates.
How it Works
The setup process gives you granular control over your testing variables and audience splits before automatically handling the evaluation. To create an A/B Test, navigate to Campaigns → A/B Tests and click Create A/B Test.
- Configure Variations: You can create exactly two variations (Variation A and Variation B) to test either your subject line or your email content body, focusing on one variable at a time.
- Audience Distribution: You have full control over the percentage split. For example, you can send Variation A to 25% of your audience, Variation B to another 25%, and reserve the remaining 50% for the winning campaign. The audience state is safely "frozen" at the time of launch so the splits remain consistent.
- Winning Metric: Choose how success is measured. You can choose between the open rate or click rate.
- Limit Values (Thresholds): You can optionally set a target threshold percentage for your metric. This represents the minimum performance level a variation must reach to be declared the winner.
Automatic Winner Evaluation
Once the test is launched, the system operates in the background to monitor performance for your specified test duration (in hours).
Declaring a Winner: If a variation reaches your optional limit value, it wins immediately. If both reach it, the higher rate wins. If the test duration elapses without hitting a limit, the variation with the highest metric value, even by a fraction of a percentage, is automatically declared the winner. In the rare event of an exact tie, Variation A wins by default.
Automatic Sending: Once a winner is declared, the system handles the rest. It automatically creates a clone of the winning campaign and sends it to the remaining portion of your audience.
Important Considerations
Please note the following constraints when configuring your A/B tests to ensure smooth delivery:
Campaigns Only: A/B testing is exclusively available for regular email campaigns. It does not apply to automated workflows or SMS messages.
No Minimum Audience: There is no minimum audience size enforced to run a test.
Test Finality: Variations can only be edited while the overall test is in a DRAFT status. Once a test status changes to RUNNING, it cannot be edited anymore.
Related Articles
How To Use Email Tagging in Marketing Campaigns
How To Use Email Tagging in Marketing Campaigns Get more bank for your email buck I'm a big believer in the power of list segmentation. I hope you are, too. Segmenting helps marketers send targeted messages that resonate with an audience. Adding tags ...Create and Send Email Campaigns
How to Create and Send an Email Campaign in TinyEmail This guide walks you through the steps to create, design, and send an email campaign using TinyEmail. Follow the instructions below to launch your first campaign quickly and confidently. Overview ...RSS to Email: Add a JSON Web Feed to Your Recurring Email Campaigns
RSS to Email: Add a JSON Web Feed to Your Recurring Email Campaigns Use the web to inject dynamic content into your messages Here's an exciting feature that offers clever marketers all sorts of product promotion possibilities. You can inject custom ...Locate Previous Campaigns
Locate Previous Campaigns Use a filter to find past work Sometimes, the volume of data email marketers manage can be overwhelming. Thank goodness for filters. That's especially true when you want to revisit a campaign that you drafted or sent days, ...Email Monetization
Email Monetization: Monetizing your email campaigns allows you to generate revenue by integrating ads into your emails. Below are two powerful platforms that help you achieve this: 1. LiveIntent LiveIntent provides unique HTML tags that enable ad ...