If you are considering running AB Tests on your JavaScript powered website a lot can got wrong. Testing is possible within a SPA, SSG, or SSR powered site, however, without some key features that are easy to overlook AB testing might not be possible.
If you pick a tool without these capabilities, you may be forced to rely on your development team to code all your tests manually, your experiment data might become invalid which will impact your success rate, or in the worst case you will buy a tool that your company can not use (I have seen this many times)
If you are thinking about starting AB testing and you use React, Vue, Angular, NextJS, Astro, Svelte, etc... this is a must read article for you. Within this guide, I will tell you about the two big reasons why your AB tests might fail and what you can do to ensure AB testing success!
Bucketing
Before we get to the good stuff, you need to be aware that when you want to implement client-side AB testing on a website, typically you will need to install a AB testing tool and its JavaScript snippet within your site.
This JavaScript snippet is the magic that will allow your editors to make changes within the AB testing tool so they can be applied on your site.
As part of creating any test, you will need to decide the rules of how people are eligible to be added. This process is known as bucketing users within an experiment. When it comes to bucketing users there are two checks that need to be made:
When the tool should check if someone should be added into an experiment
The criteria that a user must meet to be added into an experiment
These two thing are solved with different solutions but both very important for success.
Within a traditional website, as users navigate around your site they will click on links. After a link is clicked, a request is made to the server to get the requested page. As part of that request, the server will either generate that page from scratch or get a version from cache and return it to the users browser.
Bucketing users within a traditional website is easy. You can simply say when your web page loads and after the experimentation snippet loads, check if that person should be added into an experiment.
If you want to run a test within an SPA, this will not work. Within an SPA after the initial HTML and JavaScript has been downloaded from the server, the user might not need to talk to the server again. Using the traditional method would mean that you could only add someone into an experiment, the first time they land on your page and that's it.
If a tool only supports traditional bucketing methods then its rubbish. Any good AB Testing tool should provide ways to bucket someone into a test that supports the nature of how people interact within dynamic websites. The AB testing tool that I have used the most is Optimizely Web. In this tool you get several dynamic website bucketing options, these include:
To cover these options, we have:
- When the URL changes: Triggers activation each time the URL changes within the browser, even if the full page does not reload.
- When the DOM Changes: Triggers activation each time an element on the page changes, e.g. added, edited or removed
- Manually: This mode is mainly for activation when using a performance mode within Optimizely called Optimizely Edge
- When a call-back is called: Triggers activation based on a call-back function and custom JavaScript you write
- When a JavaScript condition is true: Triggers activation based on some custom code run against your page
After defining when the tool will check, the next bit is defining the rules/criteria that must be meet. These rules within Optimizely lingo are called `conditions. Optimizely supports these conditions:
- URL Targeting: Use the page URL. Here you can add rules like to match on a simple match,
- Element is present on page: Allow someone in if one or more CSS selectors exist on the page. This feature uses
document.querySelectorAll()to determine whether the element exists - Custom JavaScript: This allows a developer to create a custom rule using JavaScript to check for a certain condition on the page, e.g. value of a data layer object, etc...
As long as your tool support these types of bucketing options you should be OK! If you are reading this and your Optimizely Web customer and this doesn't work, you nee dot make sure this setting is enabled!
Targeting
The next important consideration for successful client-side AB testing within a dynamic JavaScript powered website is targeting. Within a client-0side tool you should get access to a visual editor. This editor will allow non-technical people to create tests. This is useful as it means developers are freed up to work on other stuff.
When changes are made within the editor, each change is associated to a position on your web page. The way in which that position is determined is typically via the sites HTML structure. Typically, the AB testing tool will use the elements position to determine where a change needs to be applied. This selection process is known as targeting!
The visual editor element selector might use the position of the HTML elements on the page, the class names of the elements, or, the elements ID in order to determine where a change should be applied. If your site uses something like CSS modules, where the class names are dynamically generated by the framework on build, this can be a show stopper for some AB testing tool to work correctly on your site.
If your site uses dynamic class names, you need to make sure a testing tool can support this. If it does not, you will NOT be able to use the tools visual editor!
Think about it, in this set-up its possible that the class names might change every time you release code to production. If this happens, your existing tests that are reliant on the old class name will break as soon as the code changes. When this happens your test data becomes invalidated and the tests changes wont be applied correctly, potentially breaking your pages and at a minimum making your site look unprofessional!
The way to fix this is to apply an additional ID or attribute onto your components instead. Within Optimizely, you can enable the attribute that you want to use from within the advanced tab. here you can add the name of your attribute:
After enabling this feature, within your code, for each component you will need to add the corresponding attribute and give it a unique value that will never change.
When dynamic targeting mode is activated, the snippet will look for this attribute first. If it can not find one, it will then fallback to using the class or the HTML structure instead. This type of mode is essential for any sites that use dynamic class names.
Without this type of feature, you will not be able to AB test!
In summary, if you want to run client-side tests on your website, you need to make sure you can bucket people correctly and that if you use dynamic class names the a visual editor will still work within your HTML. If you pick a tool that can not support this, you will not be able to run tests successfully. Happy Coding 🤘
