# Turning Categories into Numbers

Numbering categories such as dog, cat and rabbit 1, 2 and 3 invents an order and distances that aren't there, so one-hot encoding gives each category a column of its own, which OneHotEncoder learns from the training data, with handle_unknown='ignore' for a category it never saw, while OrdinalEncoder fits categories that really have an order, given in that order.

Journeyman · AI Basics · Python · Updated November 19, 2026

By Gregor Dzierzon

## After this lesson you can

- One-hot encode a category column by hand and with OneHotEncoder, and say what handle_unknown='ignore' does with a category it never saw
- Choose between one-hot encoding and an OrdinalEncoder with its order given, and explain why numbering categories invents an order

## Key takeaways

- A model does arithmetic, so numbering dog 1, cat 2 and rabbit 3 tells it a cat sits halfway between a dog and a rabbit.
- One-hot encoding gives each category its own column, 1 for the category the row has and 0 for the rest.
- OneHotEncoder learns the categories from the training data. With handle_unknown='ignore', a category it never saw becomes all zeros instead of an error.
- OrdinalEncoder fits categories with a real order, like small, medium and large, but only when you give it the order. Left alone, it numbers them alphabetically.

Most real tables have columns of words: a pet's species, a customer's plan, a booking's channel, a car's body type. Every model in scikit-learn does arithmetic, so those words have to become numbers before a model can learn from them, and the way you turn them into numbers can quietly teach the model something false. You'll do this in almost every project with real data.

The example is Happy Tails Pet Adoption's `pets.csv`, from *Exercise: Adoption Pipeline*. Its `species` column holds `dog`, `cat` or `rabbit`, and its `size` column `small`, `medium` or `large`.

## Numbering invents an order

The quickest idea is to number the species. Here's what a model would then be allowed to believe:

```python
codes = {"dog": 1, "cat": 2, "rabbit": 3}

print("Halfway between a dog and a rabbit:", (codes["dog"] + codes["rabbit"]) / 2)
print("A rabbit is", codes["rabbit"] / codes["dog"], "dogs")
```

```text
Halfway between a dog and a rabbit: 2.0
A rabbit is 3.0 dogs
```

A cat isn't halfway between a dog and a rabbit, and nothing is three dogs. But a linear model multiplies the code by one weight, so it would have to treat each step from dog to cat to rabbit as the same-sized change, in that order, and nearest neighbors would call a dog closer to a cat than to a rabbit. The numbers carry an order and distances the categories don't have. A column of categories with no order is a **categorical feature**, and it needs a different treatment.

## One column per category

**One-hot encoding** gives each category a column of its own, holding 1 when the row is that category and 0 otherwise. Exactly one column in each row is "hot":

| species | species_cat | species_dog | species_rabbit |
|---|---|---|---|
| dog | 0 | 1 | 0 |
| rabbit | 0 | 0 | 1 |
| cat | 1 | 0 | 0 |

No column is bigger than another, and every species is the same distance from every other. scikit-learn's `OneHotEncoder` learns the categories with `fit`, from the training data, like the scaler in *Scaling Features*:

```python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoder

pets = pd.read_csv("pets.csv")
X = pets.drop(columns=["pet_id", "adopted_in_30_days"])
y = pets["adopted_in_30_days"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)

encoder = OneHotEncoder(sparse_output=False)
encoder.fit(X_train[["species"]])
print(encoder.get_feature_names_out())

three = pd.DataFrame({"species": ["dog", "rabbit", "cat"]})
print(encoder.transform(three))
```

```text
['species_cat' 'species_dog' 'species_rabbit']
[[0. 1. 0.]
 [0. 0. 1.]
 [1. 0. 0.]]
```

It took a table (`X_train[["species"]]`, two pairs of brackets) and found the three categories, listed in alphabetical order. `sparse_output=False` asks for a plain grid you can read; without it you get a *sparse matrix*, a compact form that stores only where the 1s are, which a model reads the same way.

## A category it never saw

A ferret arrives. Nothing in the training data was a ferret, so the encoder has no column for it:

```python
ferret = pd.DataFrame({"species": ["ferret"]})
encoder.transform(ferret)
```

```text
ValueError: Found unknown categories ['ferret'] in column 0 during transform
```

That's the default, `handle_unknown="error"`. A model that stops working the day something new shows up isn't much use, so tell the encoder to let unknown categories through as all zeros:

```python
forgiving = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
forgiving.fit(X_train[["species"]])
print(forgiving.transform(ferret))
```

```text
[[0. 0. 0.]]
```

All zeros says "none of the species I know". The model still guesses, using the pet's other features. It hasn't learned anything about ferrets, so the guess leans on what the ferret shares with the pets it did see.

## When the order is real

Sizes are different: small, medium and large do have an order, and large really is more than medium. `OrdinalEncoder` turns each category into one number, 0, 1, 2 and so on. Left to itself it numbers them alphabetically, which here is backwards:

```python
from sklearn.preprocessing import OrdinalEncoder

sizes = pd.DataFrame({"size": ["small", "medium", "large"]})
alphabetical = OrdinalEncoder().fit(X_train[["size"]])
print(alphabetical.transform(sizes).ravel())

in_order = OrdinalEncoder(categories=[["small", "medium", "large"]])
in_order.fit(X_train[["size"]])
print(in_order.transform(sizes).ravel())
```

```text
[2. 1. 0.]
[0. 1. 2.]
```

`categories=` takes one list per column, so the order goes in a list inside a list. `.ravel()` flattens the one-column answer into a row so it's easier to read. Even with the right order, an ordinal code still says medium-to-large is the same step as small-to-medium; if that isn't true for your label, one-hot encoding is the safer choice. In the adoption data only large pets wait longer, so this chapter one-hot encodes `size` too.

## Do it: one-hot encode the arrivals

The front desk at Happy Tails logs each new arrival's species, and a model needs those words as one-hot rows, with the categories learned from the training pets.

The inputs:

- `train_species`: the training pets' species, one word each.
- `categories`: the categories to encode with, sorted, each once.
- `species`: the new arrivals' species, one word each.

For example:

```python
train_species = ["dog", "rabbit", "dog", "cat"]
species = ["rabbit", "cat", "ferret"]
```

| Step | Work | Result |
|---|---|---|
| Learn the categories | each species once, sorted | `["cat", "dog", "rabbit"]` |
| A rabbit | 1 in the rabbit column | `[0, 0, 1]` |
| A cat | 1 in the cat column | `[1, 0, 0]` |
| A ferret | not a category the training pets had | `[0, 0, 0]` |

So `learn_categories(train_species)` returns `["cat", "dog", "rabbit"]`, and `one_hot(species, ["cat", "dog", "rabbit"])` returns `[[0, 0, 1], [1, 0, 0], [0, 0, 0]]`.

**Your job:** write two functions.

- `learn_categories(train_species)` returns a list of the different species in `train_species`, each once, sorted alphabetically.
- `one_hot(species, categories)` returns a list with one row for each arrival: a list of 0s and 1s, one number per category in `categories`' order, 1 where the arrival's species matches the category.
- An arrival whose species isn't in `categories` gets a row of all 0s, as `handle_unknown="ignore"` does.
- With no arrivals, `one_hot` returns an empty list.

Practiced live on the lesson page: write the Python right there and check it.

## Check yourself

2 questions to try on the lesson page.

## So what

Categories become numbers without lying about order when each one gets its own column, learned from the training data and ready for a category that shows up later; an ordinal code is only for an order that's real, given by you.

## Sources

- scikit-learn 1.9.1, [OneHotEncoder](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html), API reference: `categories='auto'` determines the categories from the training data; `handle_unknown` is `'error'` by default, and with `'ignore'` an unknown category's one-hot columns are all zeros; `sparse_output=True` by default returns a SciPy sparse matrix in CSR format; `get_feature_names_out` joins the feature name and the category with `_`.
- scikit-learn 1.9.1, [OrdinalEncoder](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OrdinalEncoder.html), API reference: `categories` is `'auto'` (from the training data) or a list where `categories[i]` holds the categories expected in the ith column.
- scikit-learn, [Preprocessing data: Encoding categorical features](https://scikit-learn.org/stable/modules/preprocessing.html#encoding-categorical-features), User Guide: `OrdinalEncoder` turns each categorical feature into integers from 0 to n_categories − 1, which estimators would interpret as ordered, "which is often not desired"; `OneHotEncoder` turns a feature with n_categories values into n_categories binary features, one of them 1 and the others 0.

## Key terms

- [One-Hot Encoding](https://www.ask-codey.com/glossary/one-hot-encoding)
- [OneHotEncoder](https://www.ask-codey.com/glossary/onehotencoder)
- [OrdinalEncoder](https://www.ask-codey.com/glossary/ordinalencoder)
- [Categorical Feature](https://www.ask-codey.com/glossary/categorical-feature)

## Related

- [Scaling Features](https://www.ask-codey.com/lessons/ml-scaling-features)
- [Different Steps for Different Columns](https://www.ask-codey.com/lessons/ml-column-transformer)
- [Exercise: Adoption Pipeline](https://www.ask-codey.com/lessons/exercise-ml-adoption-pipeline)

Source: https://www.ask-codey.com/lessons/ml-encoding-categories
