How to identify which experiments to run

At the same time, I have not hesitated to say that, too often, they are carried out for the wrong reasons. We have explained how the purpose of online testing is to answer questions about how people are using your website.

But how do you know which questions to ask? And how do you know if experiments are even a viable option for answering your questions in the first place? B before you start you need to make sure that you know these things.

Not sure where and when to start? Have no fear - we are here to help. Let's go.

How to know if you can run tests

Before offering tests to run, you need to make sure that you can run them accurately. Experiments should be completely irrelevant until you have an established online presence and ways to track the behavior. To do this, you will need five things.

1) Traffic

To be sure that the results of a test are unlikely to be influenced by chance, you must have a high volume of traffic. Some experiments require larger sample sizes than others - even hundreds of thousands, in some cases - but in general you will need at least 100 unique pageviews per day to achieve statistical significance in a given case. reasonable amount of time.

2) Objectifs

In an experiment, your hypothesis is the statement you are working on. But what are you trying to i improve as a result of this test? These are your Key Performance Indicators (KPIs) - the quantifiable measures of the success of the experiment. Without these, you do not have a North Star to guide the purpose of your experience, nor the objectives behind it.

3) Monitoring

To measure and observe the performance and results of your test groups, you will need to define the data that you will be monitoring and monitoring. In the digital realm, this can include factors such as:

• What pages are visitors visiting?
• Where do they come from?
• What is it all about once they get to these pages? Do they convert, bounce, or take some other action?

4) Basic metrics

Even if you hope to make improvements to your funnelr conversion, before starting an experiment you should have a funnel conversion rate (CVR) established and recorded. In other words, before you start you should be able to follow:

Visit the funnel -> successful customer

If you try to start an experiment without this information, you won't have a benchmark to compare where you were before you ran it - and therefore you won't know whether your situation is better or worse. a result.

5) You have selected all the fruits at hand.

Make sure you have fully developed and rehearsed all the basic requirements for your funnel to function or even function properly. For example, in the ecommerce industry, you might want to do something like optimize your online product catalog. But you can't do it until you've got yourselfsure that all products are listed there, that you have a complete online payment system, and that visitors can contact you for customer service.

We have a sentence for this step: "Don't start hanging pictures before painting the walls. "

How do I know if I have these five things?

If you are wondering this question, w We recommend that you run an A / A test - an experiment in which you do all of the running and tracking moves of an experiment, without changing anything. We do it in three steps:

1. Run the dummy test for five working days.
2. Take the test.
3. Analyze the results.
• Do you have more than 500 unique users registered for the test?
• Can you follow the full funnel of both groups of tests?
• Is the funnel CVR roughly equal for the two experiment groups?

Aso do you have these five things? Nice job - you are already ahead of the curve. But yet experimentation only makes sense when you can identify questions that deserve an answer through quantitative research.

Identify experiments

First of all, you need to choose a funnel that you want to optimize through experimentation. Once you have your funnel, identify any unanswered questions you have about how your audience moves between their stages. To identify unanswered questions, we need to take stock of what we already know.

Identify who is moving through your funnel and why

Do you know exactly who is entering the funnel and from where, with both quantitative and qualitative data at the bottom support? How about why they go down the funnel, with the same support data? If you don't know the answers to these questions, this is where you should start.

Next, if you look at your funnel, can you see why people don't convert between stages?

Source: Apolline Adiju

Identify gaps in knowledge about how people move through your funnel

Consider the following conversion:

Basic visit> purchase

Our goal is to identify the reasons why people do not convert between stages of our funnel. To find out, we need to list the reasons why we think that people don't convert and search for data to back up our claims. We will know that we have listed the good reasons why we can represent more than 100% of unconverted users, with dataes to support.

• Aren't people buying because:
• They have unanswered questions about the product? (Let's say this reason represents 5% of users who don't buy.)
• Not ready to make a purchase yet? 10% of non-buyer users
• They do not see how the product fits into their life? 4 0% of users who are not buyers
• The product does not correspond to what they are looking for? 5% of non-buying users
• Are there alternatives at a lower price? 10% of non-buyer users
• Are there alternatives with more or better features? 10% of non-buying users
• They lack confidence in the product or the company that sells it? 30% non-buyer users

Note: These percentages add up to> 100% - each user given often has more than one reasons to decide not to buy.

If you find that you are struggling to come up with a list of reasons people don't convert, you will need to get some qualitative feedback from your customers.

Once you 've compiled a complete list, take a step back and look for areas of opportunity. For example, in the list above, say "They don't see how the product fits into their life" and ask "Why?" Assuming we're in line with the product market, there must be something we don't understand here. Otherwise, how 40% of non-buyers can users cannot see themselves using the product? This could become a fundamental question that we aim to answer through quantitative experimentation.

To sum up: experiences answer questions. To identify experiences, you need to identify gaps in your knowledge and, todo this, list what you do know - this will help you more easily identify what you don't know.

Next Steps

We hope this article has provided you with the tools to identify when to run experiments. In my next article, we'll walk you through how to uncover the unanswered questions about your funnel and prioritize those questions to maximize your investment in a given experience. In addition, we will provide a useful framework for doing this.

How do you identify which experiments to run? Let us know your approach - and hey, we might even feature your experience on our blog.