So, we’ve just covered what is an A/B test and what is a multivariate test, but what is an A/A test? In this quick tutorial we go over A/A testing, what it is and how to use it to help you properly conduct A/B or multivariate tests.
A/A testing is the tactic of using A/B testing to test two identical versions of a page against each other. Typically, this is done to check that the tool being used to run the experiment is statistically fair.
Hi and welcome to this quick introduction on A/A testing. So we’ve
just covered what is an A/B test and what is a multivariate test, but what is
an A/A test? As mentioned in the previous video if something is broken or the data
is messy or maybe you don’t randomize correctly this will have an impact on
your experiment you should make sure that your A/B test is being conducted
properly first by setting up an A/A test. An A/A test is when you compare the
identical experience on a different random set of users. For example your
original version, version A, your control is a green “Buy Now” button and your
alternate version, version B, your treatment, is also a green “Buy Now” button
an A/A test is used to compare the identical experience and is used for
validation of setup. It is good to set up an A/A test to show a random subset of
users the same experiment and another random group of users the same
experiment. You should have different users and different data and if it is
the same experience but the metric is different something is wrong.
So let’s say I’m running a drug test, half of the people I am testing the drug on get one
type of medication the other half can get the same type of medication. So what
kind of problems can there be if it is the same experience but the metric is
different the reaction is different something is wrong. Again once you set up
your A/A test you can then move on to conducting your
A/B or multivariate test. Thanks for watching, give us a like if you found
this useful or you can check out our other tutorials at tutorials.datasciencedojo.com
Check out our video on online experimentation:
Steps in Experimentation
What is Multivariate Testing?
More Data Science Material:
[Video] Online Experimentation and A/B Testing – Community Talk
[Video] Introduction to Online Experimentation and A/B Testing
[Blog] Ethics in Research: Conducting A/B Testing on Customers