If you are interested in learning about what's new within Optimizely Web, you are in the right place. Carry on reading to learn about the 6 greatest and latest features that have been added within Optimizely Web within 2024. This list is surprisingly long, so strap in 🔥🔥🔥
Google GA4 Events and Audiences Sync
In 2023, Google made a shock announcement that is was going to sunset its AB testing tool, Optimize. As a result of this unexpected news, many companies found themselves needing to find a replacement AB testing tool. I think its fair to say that Optimize had limited AB testing capabilities, however, it did do one thing better than any other AB testing tool and that was integrate with other Google products, specifically analytics.
The main reason for this limitation in these third-party tools was because until Google decided to sunset Optimize they didn't provide APIs for third-party companies to access certain data. Due to the announcement of this sunset and because Google and Optimizely are premium partners, Google agreed to work with Optimizely to change that.
Released within Q1 of 2024 in beta, Optimizely now support syncing audiences that are created within GA4 inside of the audience builder within Optimizely Web. Setting this new integration took me less than 5 minutes end-to-end. All you need to do is enable the integration, authenticate your account and then add your GA4 property ID and the sync just magically works.
The other data integration side within experimentation is around events/metrics. If you do reporting within GA4 to better understand your customers, how do you make sure you know which experiment they the have been exposed to? The fix to this dilemma is to sync your experiment data back into GA4. This is where the second integration comes to the rescue.
Within 2023/2024 Optimizely also created a second integration, the GA4 event export integration. Installing this extension is again simple. Verify your account and add your GA4 property ID. After doing you will see two events being fired within GA4, optimizely_decision and experience_impression.
Once this integration is enabled on an experiment, Optimizely will automatically send your variation data into GA4. Asides from seeing the generic events within GA4, you can also access more fine grained event data like the variation name a user has been exposed to. This requires a couple of extra minutes of set-up effort within GA4 however, once enabled you can create reports that show how your experiments measure up with the events you track in GA4!
Advanced Audience Targeting/Connect
The next feature that came at the start of 2023 is also another segmentation/personalisation feature, the Optimizely connect platform. The benefit of this platform for Optimizely Web customers is as a passthrough for segmentation/personalisation.
If you currently have your own CDP and you want to use that as your source of truth for personalisation/segment groups/audiences, the connect platform will allow you to sync these third-party segments from that CDP into the Optimizely Web audience builder.
Within 2023/2024, Optimizely created and released 6 connectors for this platform. These connectors will allow you to pass your audiences contained within either mParticle, ZeoTap, Segment, or Tealium Audience Steam.
Automatic Sample Ratio Mismatch
I'm guessing that most normal folks will have zero idea of what this next new feature does and that even the name sample ratio mismatch means nothing to most people (I was included in this group a few weeks ago)! If on the other hand you are a data geek/analyst then this is a feature that you will very much appreciate!
When it comes to determining if an experiment is successful or not, then in terms of data there's an anomaly called Sample Ratio Mismatch (SRM) that can negatively impact your programs overall success.
This anomaly impacts all A/B testing tools in the marketplace and can mean that a small percentage of your total experimentation results can become invalidated. Now I do not claim to be a data analysis, or a maths guru (I got a D in my GCSE maths!) and I'm not the best person to explain the maths behind this. Instead, if you want a more in-depth explanation that there's a great article here that goes into more detail written by someone who knows what they are actually talking about :P
In simple terms, when you are running a test with say a 50/50 split sometimes this distribution can fail. What could potentially end up happening is that distribution could end up looking more like a split 70/30 for example. The exact cause for why this happens is an industry mystery and causes can vary. The outcome is that when this anomaly happens within your test the underlining data can be questioned. Cause why SMR could happen include redirect errors on a page, JavaScript code issues, snippet issues, code releases etc... If it helps think of it as someone who is exposed to a test but for whatever reason their event data is lost.
When SRM happens, all tools within the marketplace basically bury their heads in the sand and pretend it didn't happen. Optimizely's AutoSRM changes that. Within 2023, the Optimizely data team created a new sequential stats algorithm to help identify in real-time tests that are being impacted by SRM. This feature is another game changing data thing that no other tool can do.
In its current form, within your results page you will see an experiment health check. Think of this check as a double-duty feature. First it confirms that your test data is valid. Second, it acts as an early flag for your team so they do no waste time waiting until the end until of the experiment to find out the bad news! Having a early warning flag, will allow to investigate the issue sooner, helping you improve your velocity!
Currently, understanding if your test is being impacted by SRM is a manual. however, when we look at the next new feature, this will soon change!
Notifications
Another editor related time-saving feature that was also released in 2024 are notifications. If like me you are always busy then you may not have the time to constantly log into Optimizely to check to see if your tests have hit statistical signific.
The good news is that you can save yourself from this constant checking, with notifications. It is now possible for you to set on an user level an email notification on any project. When enabled, you will get an email to say when a test has hit stats sig.
The other thing to note is that notifications are currently in phase one and the types of things that you can get notified about will grow over time. In the future, notifications will additionally be extended to AutoSRM notifications, notifications for feature flags and experiments that are not working as expected in real-time, and more..
Feature Collab
Within the Optimizely product suite, Optimizely sells a tool called Optimizely Content Marketing Platform (CMP). If you look within the Gartner CMP quadrant you will find that CMP has been the market leading tool within that quadrant for the last 7 years.
Within 2024, Optimizely created some experimentation specific modules within the CMP. The cool thing is that within 2024 you get free access to this tool and these modules for free within the standard tier of the experimentation platform . I say free as Optimizely did not change the price when they added this access.
These new modules combined are now known as feature collaboration. Feature collaboration allows you to better manage your entire experimentation program. In terms of capabilities, the first thing you will likely use is the idea backlog. This aim of this backlog is an area for you to store your ideas about what AB test you might run next. If you think about how you currently manage your list of what to test next, if your using email, chat, a spreadsheet or word doc, this tool will make an experimentation leads life easier on what to work on next.
After you pick an idea that you want to turn into an AB test, you can then add it to a workflow. Within the tool you get several different views to help you visualize and plan your experimentation program. These views include a calendar view, a GANTT chart, a Kanban board and a list view to see a sequential list of task that need to be followed in order to create the new test.
When you assign an idea to someone, they can use the CMP to upload their design work and get feedback from the team on it. using the customised workflows, managers can sign-off the work via email notifications without even logging into the tool. Once signed up, the idea can then be turned into an experiment!
One thing I personally would like to see in the tool is program level reporting. I know this is in the Optimizely backlog so it should come within 2024 at some point, also I know Optimizely are working on some useful templates to generate PDFs and reports on how an experiment is performing, again reducing the admin burden for testers setting up tests. For me this is cool release, as you get two best-of-quadrant tools for the price of one!
Optimizely Opal
The last feature I want to mention is Optimizely Opal. Optimizely released Opal in 2023/4 and Opal is the name for its AI initiative. Being a cynical developer by nature, when I first heard about Opal I immediately thought of this meme:
Some of Opal does definitely wrap some of the well known generative AI APIs like ChatGPT, however, Opal also covers a bunch of patented machine learning algorithms that Optimizely has built over the last 15 year as well. As Opal is so large, I created a specific video and blog post covering everything that Optimizely offers within 2024 so I recommend checking out [this] article out to get the full low-down.
In terms of new experimentation features, we get two things, Opal chat and content suggestions. Within Opal chat, you can get 24/7 help from AI about what each Optimizely feature does, or, to even get some potential hypothesis suggestions.
The second new feature is content suggestions. With content suggestions, it is now possible to click on any bit of text within the visual editor and get Optimizely to give you 5 different suggestions on how it could be re-worded. If you test the best copy to use on a page, this is a big time saver. Instead of thinking about what copy to write, instead you can create a test with the auto-suggestions and use data to prove which one is better instead!
I think we can agree Optimizely has been very busy in the last 12 months. In terms of new features, some of the things listed above are not minor things and certain features could take other companies years to build from scratch. I think this is a welcome update because in 2022/2023 Optimizely moved from AWS to Google Cloud. Due to this move we didn't see a lot of new Optimizely Web features, however, I think we can agree they smashed the releases in 2024!
Happy Coding 🤘
