The Beginner’s A/B Testing Toolkit

A/B Testing
one black shoe and one brown shoe

Source: Flickr, Alexandre Duret-Lutz

A/B testing is an integral part of conversion rate optimisation. Getting it right can be hard, after all the idea of A/B testing is trial and error. That’s why we’ve rounded up 5 great resources from around the web.

We’ve focussed on resources that make A/B testing easier at every step, from planning and building your variant page(s) to tracking and interpreting the results. We also want to show that A/B testing is accessible to marketing teams who don’t have a technical team at their disposal.

Planning Your Test

Learn from Experience with Which Test Won

Let’s assume you’re onboard with the idea of A/B testing. It makes sense to try out a couple of different things and see which one gets the biggest number of conversions but where do you start? What elements of your homepage, landing page or marketing email are worth creating A/B tests for?

The most useful page elements to experiment with include:

  1. Call to Action buttons e.g. change placement or design, especially if you’ve used a graphic that conveys meaning
    • Login and Sign up buttons
    • Buy
    • Add to Cart
    • Go to Checkout
  2. Navigation
  3. Your content offering (i.e. which products, services or blog post you highlight)
  4. Images
  5. Headlines
  6. Form Fields

These are the parts of the page that have the greatest affect on how users experience your site.

To understand more about why one version works better than another take a look at Which Test Won. This site gives members a chance to learn from the experience of others by reading up on the results of A/B tests that have already happened.

For example, this free test of the week asks you which version is better, a responsive site design or a mobile-specific site? You can then see other people’s answers and the results of the test.

Two iPhones showing website designs

The cool thing about looking through the results of tests is that you can check out what worked or didn’t work for other companies in the same industry as you. What worked with their audience could work with yours. Alternatively if you have an idea of what you’d like to change on your site you can view tests that focus on the element you have in mind.

Plan with Evan’s Awesome A/B Tools

You’ll hear it again and again, there’s no set length of time to run an a test for. Most experts will advise you to make sure you don’t run the test for too short a time. All of that sounds a bit vague. There is a more sure method of planning the length of time you should run your test for. Enter Evan Miller. Miller has developed a set of tools that help with the planning process at the start of any A/B test.

Overview of four a/b testing tools

Calculate how many visitors you’ll need to each version (aka branch) of your test:

Sample size calculator

The sample size calculator is designed to tell you how many visitors you need to each version of your test which allows you to predict how long the test will take to run. Input your goals and specify the parameters of the test.

The rest of the tools in the suite let you query your experiment, either hypothetically, aiding the planning process or with real data, helping to illustrate the results.

Getting Started

You’re ready to start; here’re the tools you’ll need to create page variants and track the results of your test.

Test Content Changes with Google Analytics Content Experiments

Added a few years ago to replace Google Website Optimizer, Content Experiments is an integrated feature of Google Analytics. It’s useful because you can set up experiments within your Google Analytics account and monitor them there.

Create a new experiment form

This tool doesn’t help you create your variant pages. You must have prepared these before you can set up a new Content Experiment. Instead it’s a tracking tool that gives you live results throughout the duration of your experiment.

Although not compulsory, the option to add goals makes Content Experiments stand out from other A/B testing tools. There are 4 types of goals you can set for your experiment:

  1. URL destination goals

  2. Event goals

  3. Session duration goals

  4. Pages per session goals

By keeping your goals at the forefront of the experiment it ensures you stay focussed and makes the results clearer.

Content Experiments report page

Anyone who already uses Google Analytics will feel right at home with the reports generated. It’s free too- winning.

Edit On-Page with Optimizely

If you’re looking for greater insight than Google can offer then it’s time to look at paid options. Optimizely is a comprehensive service that helps to create multiple variations of your web pages and provides a platform for tracking.

It’s interesting because the process of creating variations of your page to test is simplified to the level of clicking elements of the page. Add as many variations as you like and begin the experiment at the click of a button.

Formisimo homepage loaded in Optimizely

Click an element to add to, remove or change it. The ability to edit on page while the rest of your design is still visible to you shortens the process. Rather than changing code and opening your web page in a test environment to see what it looks like you can do it all on-page with instant results.

Optimizely also have a sample size calculator much like Evan Miller’s.

Build and Test Landing Pages with Unbounce

Landing pages are fantastic for conversions. Most landing pages have a single call to action (CTA) which prompts a conversion -although recent data shows that long landing pages with multiple CTAs are performing well so why not try long vs short versions?

Landing pages are a bold approach, so playing with different designs, content and CTAs is important in order to see what works.

Unbounce is a landing page builder that has a bunch of templates to choose from. Marketers and other less tech-savvy business people can produce landing pages without developers and designers.

Three templates of landing pages

The ease of creation allows you to build several landing pages for A/B testing, still without the help of a designer or developer.

Several of the tools mentioned, such as Unbounce and Optimizely, are designed to help marketing professionals conduct A/B testing without relying on designers, developers or much technical know-how. With cool resources like that there’s no excuse not to test variants. In addition, tools like the sample size calculator take the guess work out planning your tests.

image representing average form conversion rate uplift from using Formisimo From 0% to 24% Form Conversion Uplift Get the insight that increased form conversions by 24% (and rising)
Latest Insight
Improving Insurance Forms Reviewing the top 31 insurance forms