The Scientific Method of Split Testing for CRO
When you look at your website, you are content. You have a powerful mission statement, a gleaming portfolio, well-packaged descriptions of your services or products, images with impact, and multiple touch points to either make a direct purchase or connect with sales reps.
So why aren’t you meeting your conversion goals?
It could be your site isn’t as as accommodating to the user experience as you think. There may be barriers to conversion; they are obvious to the user, but not to you.
It’s time to split test to figure out the problems and create long-term solutions so you can improve your conversion rates.
What is Split Testing?
Split testing is where you select a web page and modify it to essentially create another version of itself. The change can be as simple as placing CTAs in areas where you’d like conversions to occur (e.g. your product or services pages), moving existing CTAs, condensing forms by removing fields, or reformatting page layouts so that the content is more digestible and key benefits are highlighted.
Then, in a real-time experiment, each of the versions are displayed to users at random--half of your traffic is shown the original page version (the control), while the other half are shown the modified page version (the variation).
Their engagement with each version is measured and collected in a statistical analytics dashboard, where it is analyzed. The results help determine whether the change had a negative or positive impact on your conversion goals, or whether it had none at all.
As for what you’ll test, that will largely depend on your business objectives and what you were able to discover about barriers to conversion during data gathering. So you may test:
- CTAs - Placement, wording, size
- Copywriting- Product or service descriptions, value propositions, article length (short vs. long), headlines and subheadings (e.g. keywords, phrasing)
- Forms - Length, field types, text
- Layout - Content pages, landing pages, homepage
- Pricing structures - Test your pricing tiers
- Images - Placement, size, context (e.g. Are they on brand and appropriate for the site?)
- Content length - short vs. long
Why Split Test?
Your customers and the money you generate through their purchases are what keep your business afloat. If not enough prospects are converting on your website, then you need to take a look at the user’s experience, which is ultimately impacting your conversion rates and your bottom line.
When in doubt, test it out.
A/B testing allows you to monitor user behavior to see first-hand how they are converting on your site, but more importantly, what the barriers are to purchase. You can develop hypotheses, test them, then use the results to reconfigure your website so conversions can occur at a quicker rate.
The Scientific Method of Split Testing
Like a scientist who is conducting research for a study, he and his team will collect data, identify the study’s goal, analyze the problem, develop a hypothesis, create variants, test the hypothesis by exposing members of the study group to the variants, observe and note how each segment of the group interacts/behaves with the variants, and finally draw conclusions.
Split testing follows the scientific method and offers steps to determine why conversions aren’t occurring at the rate to which you need them.
1. Look at your website data
You’ll want to gather website analytics, either through Google Analytics or another source. Scan all the data to see where barriers to conversions are. Pay particular attention to pages with the highest bounce rates, drop-off rates and conversion rates.
Another key is to collect demographics and interest reports on your users, as this can give you some idea as to whether your conversion rates are being impacted because you are attracting the wrong people to your product or service, or need to make modifications to your site to better attract them.
2. Identify your business goal
Your conversion goals are what you will use to pinpoint whether or not the variation used in the split test has a higher success rate than the original. Make your objectives simple to comprehend and measure.
So, for example, your goals may be to increase your number of sales leads by receiving more contact form submissions for your products or services.
3. Identify your key performance indicators and define your target metric
KPIs are metrics you use to measure against your objectives. Back to our previous example: If you want to increase your number of sales leads from contact form submissions, then one important KPI would be increasing the number of contact forms submitted.
Now that you know your KPI, it helps to devise a measurable action that can improve it, like placing contact forms or CTAs that lead to your contact form on pages that are highly-trafficked and known to produce conversions. Then, during the testing period, you will want to closely analyze both the control and variant, looking specifically at the pace in which contact form submissions and, as a result, sales leads are increasing or decreasing.
4. Analyze user behavior
You need to gain more insights into visitor behavior than what Google Analytics can provide. Try behavior analysis tools like heatmaps, visitor recordings, and form analysis to get real-time insights into which barriers are being hit--and where.
5. Identify the visitor’s problem
Getting real, direct feedback from your online visitors is valuable, as it will tell you with certainty why they are not completing the action that you want them to complete. Set up entry surveys to determine why they have come to your site and exit surveys to determine why they left without taking your desired step.
Send your current customers feedback surveys to learn more about who they are, what motivated them to purchase, what their perception is of the product or service, what site issues they may have run into, and where they think you could improve.
6. Develop a hypothesis
A hypothesis is simply defining why you believe the problem has occurred. If your problem is visitors are not completing the contact form, which you need in order to place a sales rep before them, one hypothesis could be people are scared off by your long form and field requirements.
7. Think of a specific split-testing variation
Brainstorm some specific split-testing variations. You problem is visitors are exiting before completely filling out your contact form, and your hypothesis is the form is too long and asks too much of your visitors.
One variation could shorten the form and only ask the most pertinent questions--information for those stripped fields can be gathered later when the sales rep and prospect are in contact, or you may find they were not necessary after all.
8. Create the variation
Use software to create a second version of what you’d like to test, which in our case is the contact form. Many testing tools contain a visual editor so modifications are easy.
Some of our favorite tools include:
9. Test your hypotheses
Once the variation is created, visitors can be randomly introduced to either the control or one of the variations. Their interaction is measured, counted, and compared to determine how each performs.
10. Observe test data and draw conclusions
Study your test results to see which version had the most conversions and highest percentage improvement. That variation wins and should be implemented.
If there is not a clear winner, say the test is inconclusive, tweak your hypothesis, and test again.
How Long Should You Test?
For the most accurate results, you will want to run one test at a time and, depending on the volume of traffic, your test should run anywhere from one week to more than one month.
Ending a test too soon can lead to skewed results because you don’t have a significant sample size to get accurate statistics on percentage improvement, conversions rates, and confidence rates (the chance a variation will outperform the original version).
Split testing takes patience, but it has a major benefit: you’ll convert customers at a higher rate once you’ve removed barriers to purchase. This will ultimately increase your ROI.
