One Versus One vs. One Versus All in Classification Models

In this quick overview, we introduce you to the concepts of one-versus-one and one-versus-all in classification. In classification models, you will often want to predict one class from another. This is called binary classification, or one-versus-one. But what if you have more than two classes to predict? This is where one-versus-all is introduced. We will explain the difference between these two classification techniques, and describe scenarios where you may want to use one over the other.

Welcome to this short introduction to one-versus-one vs. one-versus-all in classification.
In classification models, you often want to predict one class from the other, such as
predicting apples from bananas.
This is binary classification, where one class is the positive class and the other class
is the negative class.
For example, the presence of aggressive cancer versus passive cancer, or returning customers
versus new customers, or successful applicants versus non-successful applicants.
Here we have our classic one-versus-one classification.
You simply predict one class from another class, or one-versus-one.
But what if you have more than 2 classes to predict, such as many types of cancer, or
many types of customers, or different levels in a game such as easy, intermediate and advanced?
Many classification algorithms are limited to binary classification, where it only allows
one-versus-one class prediction.
You need a way of being able to predict more than 2 classes.
This is where one-versus-all is introduced.
It treats one of multiple classes, such as easy level in a game, and predicts against
all other classes, such as intermediate and advanced levels in a game.
This means the easy class is treated as the positive class, and all other classes (intermediate
and advanced) are treated as the negative class – effectively turning this into binary,
2-class classification.
But there is a problem with the one-versus-all approach to multi-class classification.
You often have imbalance in the number of examples of subjects in one class versus all
other classes.
For example, if you have 100 easy level subjects, 100 intermediate level subjects, and 100 advanced
level subjects, a one-versus-all approach says that you now have only 100 easy level
subjects versus 200 (double the amount) of all other subjects.
When there are less examples for a model to train on, the model usually doesn’t predict
as well on those examples.
If you feed your model a ton of examples of one class and a very small number of examples
of the other class – well, guess what, it’s not going to predict well on the other class.
So we go back to one-versus-one again.
It might be better to do predictions of all pair combinations of classes.
So easy-versus-intermediate, and intermediate-versus-advanced,
and advanced-versus-easy.
This way you always have an even number of examples – 100 from one class, 100 from
the other.
And that sums one-versus-one vs one-versus-all in classification.
Thanks for watching.
If you found this video useful, gives us a like.
Or, you can check out our other videos at tutorials.datasciencedojo.com.

Learn more about Classification Models:
Introduction to Classification Models
Introduction to the Confusion Matrix
Precision, Recall and F1 in Classification

Complete Series:
Data Science in Minutes

More Data Science Material:
[Video] Introduction to Text Analytics with R
[Video] Classification Models
[Blog] A Comprehensive Tutorial on Classification using Decision Trees

(454)

Rebecca Merrett
About The Author
- Rebecca holds a bachelor’s degree of information and media from the University of Technology Sydney and a post graduate diploma in mathematics and statistics from the University of Southern Queensland. She has a background in technical writing for games dev and has written for tech publications.

Avatar

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>