Send a test email to a small sample of your list, and the system will select a winner based on predefined criteria to send to all remaining recipients.
Multivariate Testing Best Practices
Minimum List Size: A list size of over 5,000 recipients is best to ensure a reliable sample size.
Target Percentage: Aim for a sample size of about 10-20% of your list to maximize the impact of the impact of the winner, which will go to the remaining 80-90% of the list.
Only Change 1 Variable in Each Variant: The goal of Multivariate Testing is to identify what works and what doesn’t. If both the Subject Line and Pre Header Text are changed, it’s impossible to know which one generated the positive, or negative results!
Creating a Multivariate Campaign
Configuration
To get started, click on Create Campaign in your Email Campaigns page. From there, you will see the Configure step of the email creation process. Under Campaign Type, choose Multivariate.
You will now see new options appear. First, fill out your From Name, Email Subject Line, Pre-Header text, and Default Customer name next to Option A if you have not done so already. Then, clicking on the “Copy Icon” and a new row will be added for additional variants.
If only 1 variable is changing, fill out row A before hitting the copy button. Row B will pre-populate with options from Row A, so you only have to change 1 field!
Multivariate Test Settings
After you fill out your additional variants, head to Multivariate Test Settings. Configure these three settings:
Test Duration: How long the test should run. We recommend at least 2 hours to determine a winner based on opens, 3-4 hours to determine a winner based on clicks, and 12 hours to determine a winner based on revenue.
Sample Size: The size of the sample for the test. The sample size should be set to give each variant a minimum of 500 recipients. For example, a list of 5,000 with 2 variants would need a smple size set to 20%.
Winning Criteria: You will want to select which variant you would like to "Win" whether it is the variant with the highest open rate, most unique clicks, or most revenue.
When you complete your settings, choose which lists to send to under Recipients. When complete, click the Next button to select your Template on the Template Step. The Template step is where you select the overall template for your email campaign.
Design Page
When designing an email for Multivariate Testing, you can switch between your variants by clicking on the lettered icons (indicated by A,B,C, etc.) to make any changes to the content. You also are able to adjust From Name, Subject Line, Pre-header Text, and Default Customer Name on the right panel.
Helpful Hint:
If you are changing content within an email, build out the A variant 1st, then add another variant. This will copy A to create variant B and prevent you from having to re-create the email from scratch.
When you are finished designing the emails, click the Next button to get to the final Review page.
Review Step
At the top of the Review Step you will see an overview of the Configuration and Multivariant Test Settings for your Email Campaign. You'll also be able to see a Variants Section which will list each of the variants and the content that is in each of the test fields. Finally, you'll see the CAN-SPAM checklist for each variation.
Once you review your work, you are able to schedule out the campaign or send the campaign right away!
Campaign Page
Curious how your variants performed? The Campaign Page will show each variant nested under your main campaign. By clicking on the green expansion arrow to the left of the Campaign, you can see all of the variants along with their metrics!
