Always heard about the added value of A/B testing, but don’t know where to start of what to look out for? Today is your lucky day!

In this piece, we will talk about the basic setup you will need to get started with A/B testing.

1. Get your data right

Over the years I have experienced so many start-ups or small businesses who have been running on incomplete data. Getting your data right is not only the basics of doing business these days but especially crucial for getting started with A/B testing. If you don’t trust the numbers you are looking at on a daily basis get it fixed.

You don’t need an advanced set of analytics to get started with A/B testing as long as you know enough about what is happening on your platform and your goals.

Action; Get an basic analytics audit, it will be the best investment you have ever made!

2. Determine your way of testing

Data is everything and with A/B testing you want a lot of it. To get a valid A/B test up and running you need a certain amount of traffic to run your experiments. I like using this calculator to determine the exact amount of visitors ;

A/B Test Size Calculator - ABTestGuide.com

Calculate how many visitors you need for your A/B-test

abtestguide.com

What if you don’t meet the required amount of visitors?
Try and invest a bit into getting more visitors to your test using paid performance marketing. If you take to long to complete an A/B test the chance of your data getting influenced by external factors like seasonality or human interference like changing a banner is more likely.

Look, this works great in theory, but in the real world there will be cases you are just not able to get the amount of data you are looking for. This doesn’t mean you shouldn’t be running A/B tests, this just means that you need to keep this factor in mind.

For example; A B2B software company might never receive the amount of data they need to run a significant A/B test because it is a small market. But you have to start somewhere.

3. Where to test first

Ones you know your data is accurate you get to see immediately where the best opportunities for A/B testing are. Big drop-off at check-out or starting at your landing page on desktop or mobile? Now do take into consideration that some points in your customer journey are expected to perform lower than other parts of your customer journey.

To give you an example of this, the homepage of an basic e-commerce store is expecting to have a lower drop-off than your product page. This due to the goal of the customer journey point. On a product page, you either buy or leave, which is different on your homepage.

Action; Pick the biggest drop-off point and determine the top 5 tests for that customer journey point (don’t worry, we will go over what those 5 tests should be and how to get them)

4. What to test first

Your resources are limited, the amount of data you can use for your A/B tests is limited and most of all your time to invest in A/B tests is limited. So not only the location of your A/B tests is important, but also what you will be testing is important. Here are a couple of ways to find out what to test first;

Ask your customers

In the end, you are testing to help your customers, so why not ask them? Ones you determined your biggest drop off point, ask a question if a customer is stuck or about to leave using tools like hotjar or mopinion. Even tools like Intercom might provide you useful insights on what to test first.

If these tools aren’t at your disposal, just invite a couple of potential customers to join a survey and watch them go through your customer journey.

Research competitors, but don’t copy

You have been on your competitor's website, just be honest. Compare your drop-off point with your competitor’s page and look for differences. Don’t simply copy everything and paste it on your platform. You don’t know if it has been tested and you don’t know if it will perform on your platform.

Take a look at other similar websites, what works on a travel website, might work in e-commerce as well.

Effort versus (expected) impact

As I said earlier, your resources are limited, so make them count! Creating a list of variables that determine your A/B test effort and impact will help you prioritize your experiments. Just create a simple sheet with all your experiments and score each variable on a scale from 1 to 10. The experiment with the highest score wins.

This also takes out a lot of discussing with your co-workers and forces them to think about experiments in a different way.

5. Setup a proper A/B testing briefing

Having a proper A/B test briefing will help you structure your experiment workflow in the long run and make sure if you invite other people to come up with experiments they have the same mindset. Include elements like the location of the experiment, the goal of the experiment and expected effort.

Looking for a template? just ping me

6. Setup a proper A/B testing documentation

You are about to invest a lot of time and resources and A/B testing and the result is that you will get smarter and reaching your goals every day. But people forget experiment learnings or even experiments at all. So make sure to document your experiment results so you can always look at previous experiment results.

There are a lot of different solutions to documenting your experiment results, but starting out with a simple Google drive with a Google doc is just fine, to begin with. When you are starting to have a lot of experiments, search functions, time management, and team collaboration will get more important.

7. Build up enthusiasm

There is a big chance you won’t be building these experiments by yourself. Managing design, development, data analytics all in one person can get challenging.

So build experiments with a team is not only more effective but also helps you to challenge your bias at any time when you incorporate their opinions as well.

But keeping your team and the business that you work for motivated is essential. Here are a couple of ways to get people pumped about A/B testing;

Talk money

Every experiment can have a different goal from improving conversion to improving click-through, but everybody talks money. Calculate the effect of your experiment to the amount of money you are gaining (or saving but not implementing the experiment). The amount speaks more to your team than the improvement in conversion.

Want to know how to calculate this number? just ping me

Get results asap

If you start with A/B testing, start with the low hanging fruit and show your team the results asap (don’t put the result over a valid experiment). But something about getting that first experiment running gives your team this snowball effect and excited about the next one.

Share your results in a fun way

Collect a couple of experiments and bundle them into an event where your team can vote on which version they think will be more effective. Get some beers, get a price for the overall winner and have a fun moment with your team. This will not only boost team morale but also get them excited about A/B testing. Set up a contest on who will get the most effective next experiment so you can use your company as a source of new experiments.

I hope I gave you some insights on how to get started with A/B testing. In retrospect, I do see that I left a lot of blanks on how to actually execute each step. So if you feel like you have any questions or feedback, just shoot me a ping.