# Scaling Features

Features measured in different units (pounds, years, a 0 or 1) pull unevenly on any model that measures distance or penalizes large weights, so standardizing rescales each one with the training data's mean and standard deviation, which StandardScaler learns with fit on the training part and applies with transform to both parts.

Journeyman · AI Basics · Python · Updated November 19, 2026

By Gregor Dzierzon

## After this lesson you can

- Standardize a number by subtracting the training mean and dividing by the training standard deviation, by hand and with StandardScaler
- Explain which models care about the scale of features, and why a scaler is fit on the training data only

## Key takeaways

- When features have different units, the one with the biggest numbers wins any distance. Twelve pounds outweigh ten years.
- Standardizing turns a value into how many standard deviations it sits from the mean, the value minus the training mean, divided by the training standard deviation.
- StandardScaler learns the mean and standard deviation with fit on the training part only, then transform applies those same numbers to the training and the test part.
- Nearest neighbors and logistic regression care about scale. A decision tree doesn't, because it asks one feature at a time.

A pet adoption center wants to know which new arrivals may wait a long time for a home, so it can feature them early. Its records mix pounds, years and yes-or-no answers, and many models can't tell that 12 pounds and 10 years aren't the same size of difference. You'll rescale features like these whenever a model compares examples by distance or keeps its weights small, which covers some of the most common models you'll use.

The example for this chapter is Happy Tails Pet Adoption: 800 pets in `pets.csv`, our own made-up data, with each pet's species, age, weight, size, color, whether its page has a photo, and whether it was adopted within 30 days. The file comes with *Exercise: Adoption Pipeline*, beside the seed script that made it. Each code block is one notebook cell, run in order.

## When pounds drown out years

*Distance Between Examples* measured how alike two examples are with a straight-line distance. Here's a new arrival, a 1-year-old dog of 40 pounds with a photo on its page, and two pets the center already knows: a 1.2-year-old of 28 pounds with a photo, and an 11-year-old of 41 pounds without one. Which one is it most like?

```python
import math

new_pet = (1.0, 40, 1)
pets = {
    "a 1.2-year-old, 28 lbs, with a photo": (1.2, 28, 1),
    "an 11-year-old, 41 lbs, no photo": (11.0, 41, 0),
}

for name, pet in pets.items():
    print(name, "is", round(math.dist(new_pet, pet), 2), "away")
```

```text
a 1.2-year-old, 28 lbs, with a photo is 12.0 away
an 11-year-old, 41 lbs, no photo is 10.1 away
```

The distance says the old dog is the closer match, because 12 pounds of difference count for more than 10 years and a missing photo together. That's an accident of units: weigh the pets in ounces and weight would win by even more. Nearest neighbors would ask the wrong pets for their vote.

## Standardizing: distance from the usual, in spreads

The fix puts every feature on the same footing. For each feature, take the value, subtract the feature's **mean**, and divide by its **standard deviation** (how far values typically sit from the mean). The result says how many standard deviations a value sits above or below the usual: 0 is average, 1 is one spread above it, −1 one below. This is **standardizing**, and your textbook may call the result a *z-score*: z = (x − mean) ÷ standard deviation, read "z equals x minus the mean, over the standard deviation".

scikit-learn's `StandardScaler` does it for every column. For now this lesson keeps the 760 pets whose weight was recorded; *Missing Values in a Model* deals with the 40 blanks.

```python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

pets = pd.read_csv("pets.csv")
known = pets.dropna(subset=["weight_lbs"])
features = ["age_years", "weight_lbs", "has_photo"]
X = known[features]
y = known["adopted_in_30_days"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)

scaler = StandardScaler()
scaler.fit(X_train)
for name, mean, spread in zip(features, scaler.mean_, scaler.scale_):
    print(name, "mean:", round(mean, 2), "standard deviation:", round(spread, 2))
```

```text
age_years mean: 3.85 standard deviation: 2.83
weight_lbs mean: 23.26 standard deviation: 24.1
has_photo mean: 0.7 standard deviation: 0.46
```

`fit` learned each column's mean (`mean_`, "mean underscore") and standard deviation (`scale_`) from the 570 training pets. Its standard deviation divides by the count, like `statistics.pstdev`, not by one less, like `statistics.stdev`; with hundreds of rows the two barely differ. Now the three pets, measured in spreads:

```python
import math

three = pd.DataFrame([[1.0, 40, 1], [1.2, 28, 1], [11.0, 41, 0]], columns=features)
scaled = scaler.transform(three)
print("To the 1.2-year-old:", round(math.dist(scaled[0], scaled[1]), 2))
print("To the 11-year-old:", round(math.dist(scaled[0], scaled[2]), 2))
```

```text
To the 1.2-year-old: 0.5
To the 11-year-old: 4.15
```

Ten years is more than three standard deviations of age, and 12 pounds is half of one standard deviation of weight. In spreads, the young dog is clearly the closer match.

![The same three pets measured twice. In pounds and years, the new 1-year-old dog sits 12.0 from the 1.2-year-old and 10.1 from the 11-year-old, so the old dog looks closer; after standardizing, the 1.2-year-old is 0.5 away and the 11-year-old 4.15 away](https://fsdtrainingmedia.blob.core.windows.net/public/images/2026/11/a69786245efa487a972a39e9d685c4f6.webp)

## Fit on training, transform both

The scaler learned its numbers from the training pets only, and `transform` measures every pet, training or test, with those same numbers. The test pets come out close to a mean of 0 and a standard deviation of 1, but not exactly, which is how it should be: they're new pets measured with the training ruler.

```python
train_scaled = scaler.transform(X_train)
test_scaled = scaler.transform(X_test)
print("Test means:", test_scaled.mean(axis=0).round(2))
print("Test standard deviations:", test_scaled.std(axis=0).round(2))
```

```text
Test means: [ 0.09  0.08 -0.16]
Test standard deviations: [1.07 1.02 1.06]
```

Fitting the scaler on all 760 pets would let the test pets' ages and weights shape the ruler, so facts about the test part would leak into training. `fit_transform(X_train)` fits and transforms the training part in one call; the test part only ever gets `transform`.

## Which models care

Nearest neighbors, scored on the same test pets with and without scaling:

```python
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier

raw = KNeighborsClassifier(n_neighbors=15).fit(X_train, y_train)
scaled_knn = KNeighborsClassifier(n_neighbors=15).fit(train_scaled, y_train)
print("Neighbors, raw units:", round(raw.score(X_test, y_test), 3))
print("Neighbors, standardized:", round(scaled_knn.score(test_scaled, y_test), 3))

tree_raw = DecisionTreeClassifier(max_depth=3, random_state=0).fit(X_train, y_train)
tree_scaled = DecisionTreeClassifier(max_depth=3, random_state=0).fit(train_scaled, y_train)
same = (tree_raw.predict(X_test) == tree_scaled.predict(test_scaled)).all()
print("Tree, same guess for every test pet:", same)
```

```text
Neighbors, raw units: 0.679
Neighbors, standardized: 0.758
Tree, same guess for every test pet: True
```

Neighbors improved because age and the photo finally count. The tree gave the same guess for every test pet: each of its questions looks at one feature ("is age at most 4.5?"), and stretching a column moves the threshold but not which pets fall on each side. **Logistic regression** cares too, in a different way: `C` keeps all its weights small by the same rule, which is only fair when features share a scale, and its search for the weights finishes in fewer steps on scaled data.

`MinMaxScaler` is the other common choice: it squeezes each feature into 0 to 1 using the training minimum and maximum. Either way, the scaler learns from the training data and nothing else.

## Do it: standardize a new pet's weight

A rabbit has arrived at Happy Tails, and its weight has to be measured with the same ruler as the training rabbits' before a model can use it.

The inputs:

- `train_weights`: the training rabbits' weights in pounds.
- `value`: one weight to standardize.
- `new_weights`: several new rabbits' weights.

For example, with eight training rabbits:

```python
train_weights = [2, 4, 4, 4, 5, 5, 7, 9]
```

| Step | Work | Result |
|---|---|---|
| Mean | (2 + 4 + 4 + 4 + 5 + 5 + 7 + 9) ÷ 8 = 40 ÷ 8 | 5.0 |
| Squared distances from the mean | 9, 1, 1, 1, 0, 0, 4, 16 | total 32 |
| Standard deviation, dividing by the count | √(32 ÷ 8) = √4 | 2.0 |
| A new rabbit of 8 pounds | (8 − 5.0) ÷ 2.0 | 1.5 |

So `training_numbers(train_weights)` returns `(5.0, 2.0)`, `standardize(8, 5.0, 2.0)` returns `1.5`, and `standardize_all([8, 3], train_weights)` returns `[1.5, -1.0]`.

**Your job:** write three functions.

- `training_numbers(train_weights)` returns a pair: the mean, then the standard deviation that divides by the count (`statistics.pstdev`, as `StandardScaler` does).
- `standardize(value, mean, spread)` returns the value minus the mean, divided by the spread.
- `standardize_all(new_weights, train_weights)` returns a list with each new weight standardized by the training rabbits' mean and standard deviation, never the new rabbits' own.

Practiced live on the lesson page: write the Python right there and check it.

## Check yourself

2 questions to try on the lesson page.

## So what

A model that measures distance or keeps its weights small only treats features fairly when they share a scale, and the scale has to come from the training data alone, so new data is measured with the same ruler.

## Sources

- scikit-learn 1.9.1, [StandardScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html), API reference: the standard score z = (x − u) / s, where u and s are the mean and standard deviation of the training samples; `fit` computes them and `transform` uses them on later data; `mean_` and `scale_`; a biased estimator for the standard deviation, equivalent to `numpy.std(x, ddof=0)`; and that the L1 and L2 regularizers of linear models assume features centered around 0 with variance in the same order, so a feature with a much larger variance can dominate.
- scikit-learn, [Preprocessing data: Standardization](https://scikit-learn.org/stable/modules/preprocessing.html#standardization-or-mean-removal-and-variance-scaling), User Guide: standardization is a common requirement for many estimators; the scaler computes the mean and standard deviation on a training set to re-apply the same transformation to the testing set; `MinMaxScaler` scales features to lie between a minimum and maximum, often 0 and 1.
- scikit-learn 1.9.1, [MinMaxScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html), API reference: X_std = (X − X.min) / (X.max − X.min), `feature_range` (0, 1) by default, and `fit` computes the minimum and maximum used for later scaling.
- scikit-learn, [Decision Trees](https://scikit-learn.org/stable/modules/tree.html), User Guide: trees require little data preparation, while other techniques often require data normalization.
- scikit-learn, [Cross-validation: evaluating estimator performance](https://scikit-learn.org/stable/modules/cross_validation.html), User Guide: preprocessing such as standardization should be learned from a training set and applied to held-out data.
- Python documentation, [statistics](https://docs.python.org/3/library/statistics.html): `fmean` gives the arithmetic mean and `pstdev` the population standard deviation.

## Key terms

- [Feature Scaling](https://www.ask-codey.com/glossary/feature-scaling)
- [Standardization](https://www.ask-codey.com/glossary/standardization)
- [StandardScaler](https://www.ask-codey.com/glossary/standardscaler)
- [Feature](https://www.ask-codey.com/glossary/feature)

## Related

- [Turning Categories into Numbers](https://www.ask-codey.com/lessons/ml-encoding-categories)
- [Pipelines: Every Step, Every Time](https://www.ask-codey.com/lessons/ml-pipelines)
- [Exercise: Adoption Pipeline](https://www.ask-codey.com/lessons/exercise-ml-adoption-pipeline)

Source: https://www.ask-codey.com/lessons/ml-scaling-features
