Online Experimentation and A/B Testing
In this meetup, we provide a quick introduction to online experimentation and A/B testing.
To keep the tutorial self-contained, we will first give an overview of stats fundamentals needed to understand A/B testing. We then explain how A/B testing is done in an online business. We will conclude by mentioning some of the pitfalls that are common when running online experiments.
What is A/B Testing?
· A/B testing on web traffic
· A/B testing on newsletter and email
· A/B testing vs. multivariate testing
Online Experimentation Fundamentals
· Understanding the terminology. Control, treatment, factor, level, A/B, A/A
· Null and alternate hypothesis. Type I and Type II error
· Power and sample size selection. Confidence intervals
· Metrics for evaluation and categories
Next community talk:
Building Robust Machine Learning Models
Previous community talk:
Feature Engineering for Bot Detection
More Data Science Material:
[Video] A/B Testing in Minutes
[Video] Steps in Experimentation
[Blog] Ethics in Research: Conducting A/B Testing On Customers