In this tutorial, you will learn how to run A/B tests on pages powered by Episerver CMS. As of Episerver 10.1, it's now possible for you to start A/B testing your Episerver pages within the editor. A/B testing, also known as split testing, is a technique of showing two different variations of your web pages to your users and testing which one converts more customers. Split testing works by randomly assigning and displaying variants to a user and then measuring how users interact with those variations. For example, you might want to test which variant converts more clicks. A test might be made that counts total button clicks on each variation. After a period of time, comparing the total button clicks between the two variants will allow you to know which version is the best one. After a successful test, you can publish the winning variation and delete the less popular one! Improving your site using statistics can help to dramatically improve your sales funnel. The good news.. setting this up in Episerver is really easy! Want to learn how then read on ️🔥️🔥️🔥
How To A/B Test Your Episerver Page
First, open the editor and find a page that you want to perform a test on:
Make some content amends as normal. When you are finished making changes, in the publishing options drop-down (located in the top right-hand side of the main navigation panel), select A/B Test Changes. This will load the 'A/B' testing screen:
The first thing you will be presented with is a comparison view of the original page and the new variant. To start a test you can click on the Start Test button, located at the top right-hand side of the main content panel. The test will fail to start unless you set a conversion goal. This can be done in the test options section.
Before we get into the configuration details, it's probably a good time to talk about statistical significance. Statistical significance is the figure you need to reach to prove that the difference in conversion rates between a variant and the baseline is not due to random chance. You should keep this fact in mind when you run tests because to reach good statistical significance, your tests need to be viewed by enough people. If you run tests for a few hours, for example, you will never reach statistical significance. Statistical significance may come after days or months, depending on your traffic. Now you understand 'statistical significance', let us look at the different testing options:
The testing options will help you focus your tests and help you reach statistical significance. There are three main testing options. Let us drill down into each one:
Conversion Goal: This is the action the visitor must perform in order for the test to be proven. Adding new goals will require a developer to write some code, so if you want a specific conversion goal, be warned that it might involve customer development work. At the time of writing, the 'Landing page goal' is the only goal included with Episerver out-of-the-box, however, I am sure that will change in the future.
When the landing page goal is set, Episerver will class the test a success if the site visitor visits the chosen page.
Participation percentage: This is the total percentage of people who will be shown the test variation. Setting it at 100% will mean, a 50/50 test. 100% is the best option for reaching statistical significance the fastest, however, you may not want everyone to participate in your test. If you're doing the test on the homepage, for example, you might only want to run your test against10% of the people who visit your website. If the test has a very negative impact on the conversion rates you might not want to risk all of your traffic!.
These types of trade-offs are the ones that you need to consider when you start A/B testing. The more people you send to your test, the quicker you will know the outcome. Running tests may worsen your conversion rates impacting sales, so it's up to you to find the right balance. From my personal experience, I've usually gone for 100%, however, the decision is all yours 🤔.
Test Duration: Defines how long the test will run. Based on my previous experience, I would say 30 days is a minimum unless you have an extremely high traffic website. The more people you can participate in your test every day, the more you can trust the result.
A/B Testing Takeaway
A/B testing is an important tool to help improve your sales funnel and improve conversion. As I'm hoping you can see, A/B testing in Episerver is super simple. You make your changes as normal, but, instead of publishing the results, you publish them as an A/B test instead. Happy Coding 🤘
