Kinds of Machine Learning
Supervised learning learns from examples that come with answers, regression when the answer is a number and classification when it's a category, while unsupervised learning finds groups in examples with no answers, and reinforcement learning, learning by trial and reward, is outside this course.
Apprentice · AI Basics · Python · Updated
After this lesson you can
- Tell supervised learning from unsupervised learning by whether the examples come with a label
- Choose regression, classification or clustering for a problem, by what it asks for
Key takeaways
- Supervised learning learns from examples that come with their answers, the label column.
- Regression guesses a number, such as loaves sold. Classification guesses a category, such as busy or not busy.
- Unsupervised learning has no label column. Clustering finds groups of similar examples, and a person still has to say what each group means.
- Reinforcement learning learns by trying actions and getting rewards. It isn't part of scikit-learn or this course.
Sunrise Bakery's owner has more than one question. How many loaves for tomorrow? Will tomorrow be a busy day? And who are the bakery's regulars: are there a few different kinds of customer? All three sound like "machine learning", but they need different kinds of it, and naming the kind is the first decision in any project, at work or in a class assignment. It decides which models you can use and how you'll check them.
The bakery from Guessing from Examples is the example again.
Examples with answers: supervised learning
For the first question, the bakery's records hold 120 days, and every one comes with its answer: the loaves_sold column, the label. A model can study the days and their answers together, the way a student studies worked examples with the answers printed beside them. Learning from examples that come with their answers is called supervised learning: the answers "supervise" the learning by showing what a right guess looks like.
Supervised learning splits by what the answer is:
- Regression guesses a number: how many loaves, how many minutes until a delivery arrives, what a home will sell for. The answer can be any amount, and a guess can be close or far.
- Classification guesses a category, one of a fixed set of answers: busy day or not, spam or not spam, which fruit is on the conveyor belt. A guess is either the right category or the wrong one.
The bakery's second question is classification. If the owner only needs to know "busy (120 loaves or more) or not busy?", the label becomes one of two categories, even though it was worked out from a number. The question being asked, not the data, decides which kind it is.
Examples without answers: unsupervised learning
The third question is different. Say the bakery's loyalty cards record each member's visits per month and average spend. There's no column saying "this person is a morning commuter": nobody wrote the answers down, because nobody knows them yet. Learning from examples with no label is called unsupervised learning.
Its most common job is clustering: finding groups of examples that are like each other, such as members who come often and spend a little, and members who come rarely and spend a lot. A clustering method hands back group numbers, not names. A person still has to look at each group and say what it means, and whether it means anything at all: the groups describe this data, and nobody told the method what a "kind of customer" is.
Learning by trying: reinforcement learning
There's a third family you'll hear about. Reinforcement learning learns by trying actions and getting a reward or a penalty for each, like a program that learns a game by playing it thousands of times and keeping the moves that win. It doesn't learn from a table of past examples at all. scikit-learn leaves it out on purpose, and so does this course.
Which kind is it?
Two questions sort almost every problem. Does the data have a column with the answers? If not, it's unsupervised. If it does, is the answer a number or a category?

| The problem | Answers in the data? | Kind |
|---|---|---|
| Guess tomorrow's ice cream sales from the weather | Yes: past days' sales, a number | Regression |
| Sort support emails by topic, from 50,000 emails people already sorted | Yes: each email's topic, a category | Classification |
| Group a grocery's shoppers by what they buy | No: nobody has named the groups | Clustering |
| Guess whether a library member will renew this year | Yes: who renewed before, yes or no | Classification |
| Guess how many minutes a pizza delivery will take | Yes: past deliveries' minutes, a number | Regression |
In your textbook, these may be called supervised learning (with its two tasks, regression and classification) and unsupervised learning, and the label may be called the target or the dependent variable. scikit-learn's own words are a regressor for a model that guesses a number, a classifier for one that guesses a category, and a clusterer for one that finds groups. Module 2 of this course is regression, Module 3 classification, and Module 5 clustering.
Check yourself
2 questions to try on the lesson page.
So what
Before any model, ask whether the data holds the answers, and whether the answer is a number or a category: that one decision tells you whether you need regression, classification or clustering.
Sources
- scikit-learn, Glossary of Common Terms and API Elements: supervised learning has the expected prediction (the label) for each sample when the model is fitted; unsupervised learning doesn't, as in clusterers; a regressor predicts continuous values, a classifier one of a finite set of values, and a clusterer is an unsupervised predictor with a finite set of output values; the target is also known as the dependent variable, the outcome or the label.
- scikit-learn 1.9.1, Frequently Asked Questions, Why is there no support for deep or reinforcement learning?: reinforcement learning doesn't fit scikit-learn's design and is out of its scope.