What is Multivariate Testing?
Multivariate testing gives your store the ability to test certain variables within an email and automatically send the best performing email to your customers. Send a test email to a small sample of your list, and the system will select a winner based on predefined criteria to send to all remaining recipients.
Creating a Multivariate Campaign
To get started, click on Create Campaign in your Email Campaigns page. From there, you will see the Configure step of the email creation process. Under Campaign Type, choose Multivariate.
You will now see new options appear. First, fill out your From Name, Email Subject Line, Pre-Header text, and Default Customer name next to Option A if you have not done so already. Then, clicking on the “Copy Icon” and a new row will be added for additional variants.
If only 1 variable is changing, fill out row A before hitting the copy button. Row B will pre-populate with options from Row A, so you only have to change 1 field!
Multivariate Test Settings
After you fill out your additional variants, head to Multivariate Test Settings. Configure these three settings:
Test Duration: How long the test should run. We recommend at least 2 hours to determine a winner based on opens, 3-4 hours to determine a winner based on clicks, and 12 hours to determine a winner based on revenue.
Sample Size: The size of the sample for the test. The sample size should be set to give each variant a minimum of 500 recipients. For example, a list of 5,000 with 2 variants would need a smple size set to 20%.
Winning Criteria: You will want to select which variant you would like to "Win" whether it is the variant with the highest open rate, most unique clicks, or most revenue.
When you complete your settings, choose which lists to send to under Recipients. You can also use the optional Filters section to send more tailored messaging by sending to specific segments. Learn more about segmentation here!
When complete, click the Next button to select your Template on the Template Step. The Template step is where you select the overall template for your email campaign.
After choosing your template, it is time to design your email campaign! To learn more about designing the content of an email campaign in Springbot, read here.
When designing an email for Multivariate Testing, you can switch between your variants by clicking on the lettered icons (indicated by A,B,C, etc.) to make any changes to the content. You also are able to adjust From Name, Subject Line, Pre-header Text, and Default Customer Name on the right panel.
When you are finished designing the emails, click the Next button to get to the final Review page.
At the top of the Review Step you will see an overview of the Configuration and Multivariant Test Settings for your Email Campaign. You'll also be able to see a Variants Section which will list each of the variants and the content that is in each of the test fields. Finally, you'll see the CAN-SPAM checklist for each variation.
Once you review your work, you are able to schedule out the campaign using the blue calendar icon in the bottom right corner or send the campaign right away!
Curious how your variants performed? The Campaign Page will show each variant nested under your main campaign. By clicking on the green expansion arrow to the left of the Campaign, you can see all of the variants along with their metrics!