# Reading Coefficients and Trees

A linear regression trained on standardized features gives coefficients you can compare, each the change in the guess for one standard deviation more of its feature with the others held the same, but features that move together share and swap credit, a small tree's top splits show the questions it asks first, and neither a coefficient nor a split is a cause.

Journeyman · AI Basics · Python · Updated November 19, 2026

By Gregor Dzierzon

## After this lesson you can

- Read coefficients on standardized features as the change in the guess for one standard deviation more, with the other features held the same
- Explain how features that move together share and swap credit between coefficients, and why a coefficient is not a cause
- Read a decision tree's top splits with export_text as the questions it asks first, and say what they don't show

## Key takeaways

- On standardized features, a number feature's coefficient is the change in the guess for one standard deviation more, with the other features held the same, so the number features can be compared.
- A one-hot column's coefficient, with one category dropped, is the difference from that dropped category, in the label's own units.
- Features that move together share and swap credit. Leave one out, and the other takes its credit.
- A small tree's top split is the question that cut its training error the most. A feature it never asks about may still matter.
- A coefficient or a split says what the model learned from its data, never what causes the label.

Greenway Home Energy's team trained a linear regression on 560 homes' bills, and before any estimate goes out, the manager asks what it learned. A straight-line model and a small tree are the two kinds of model you can read directly, so reading them is where an explanation starts, and the misreadings people make here end up in reports and decisions.

The example is Greenway's `home_energy.csv`, from *Exercise: Explain the Energy Bill*. Each code block is one notebook cell, run in order: the first one loads the data and trains the model.

## Coefficients you can compare

*More Than One Feature* showed the trouble with raw coefficients: dollars per square foot and dollars per person can't be compared by size. The fix is to standardize the number columns first (*Scaling Features*): then each one's coefficient is the change in the guess for **one standard deviation more** of that feature. Your textbook may call them *standardized coefficients*.

```python
import pandas as pd
from sklearn.compose import make_column_transformer
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

homes = pd.read_csv("home_energy.csv")
train, test = train_test_split(homes, test_size=0.2, random_state=42)

features = ["size_sqft", "year_built", "occupants", "insulation", "heating", "has_pool"]
prep = make_column_transformer(
    (StandardScaler(), ["size_sqft", "year_built", "occupants"]),
    (OneHotEncoder(categories=[["good", "fair", "poor"], ["heat pump", "gas", "electric"]],
                   drop="first"), ["insulation", "heating"]),
    remainder="passthrough",
    verbose_feature_names_out=False,
)
model = make_pipeline(prep, LinearRegression())
model.fit(train[features], train["monthly_bill"])

names = model[0].get_feature_names_out()
for name, coef in zip(names, model[-1].coef_):
    print(f"{name:18} {coef:7.2f}")
```

```text
size_sqft            22.63
year_built           -0.63
occupants            10.39
insulation_fair      15.64
insulation_poor      39.85
heating_gas           9.89
heating_electric     23.96
has_pool             27.96
```

The categories are one-hot encoded (*Turning Categories into Numbers*), with each list of categories given in order and `drop="first"`, which leaves out the first one (good insulation, a heat pump). Each other category's coefficient is then its difference from the one left out: a poorly insulated home's guess is $39.85 more than a well-insulated home's, everything else the same. `remainder="passthrough"` keeps `has_pool` as it is, 1 or 0, and `verbose_feature_names_out=False` keeps the names plain (*Different Steps for Different Columns*).

One standard deviation is a different amount for each feature:

```python
print(train[["size_sqft", "year_built", "occupants"]].std().round(1))
```

```text
size_sqft     516.4
year_built     21.3
occupants       1.4
dtype: float64
```

So a home about 516 square feet bigger gets $22.63 more, with the other features held the same; about 1.4 more people, $10.39 more; about 21 years newer, $0.63 less, which is close to nothing. The seed script planted $40 for poor insulation, $25 for electric heat and $30 for a pool, and the model came close from 560 bills.

## Features that move together swap credit

Newer homes in this town have better insulation:

```python
print(train.groupby("insulation")["year_built"].mean().round())
```

```text
insulation
fair    1984.0
good    2000.0
poor    1978.0
Name: year_built, dtype: float64
```

Now train the same model without the insulation column:

```python
without = ["size_sqft", "year_built", "occupants", "heating", "has_pool"]
prep_without = make_column_transformer(
    (StandardScaler(), ["size_sqft", "year_built", "occupants"]),
    (OneHotEncoder(categories=[["heat pump", "gas", "electric"]], drop="first"), ["heating"]),
    remainder="passthrough",
    verbose_feature_names_out=False,
)
model_without = make_pipeline(prep_without, LinearRegression())
model_without.fit(train[without], train["monthly_bill"])
names = model_without[0].get_feature_names_out()
for name, coef in zip(names, model_without[-1].coef_):
    print(f"{name:18} {coef:7.2f}")
```

```text
size_sqft            22.94
year_built           -7.92
occupants            10.94
heating_gas           8.69
heating_electric     25.02
has_pool             29.22
```

The year built went from −0.63 to −7.92. With insulation gone, the year is the only column that carries part of what insulation said, so it takes insulation's credit. When features move together, statisticians call it **multicollinearity**, and their coefficients share credit and can swing when a feature is added or left out. Neither number is "what the building year does to a bill": the seed script gave the year no effect at all.

## A tree's top questions

A decision tree is read by its questions. `export_text` prints them; this tree is kept to two levels so it stays readable:

```python
from sklearn.tree import DecisionTreeRegressor, export_text

tree_prep = make_column_transformer(
    (OneHotEncoder(), ["insulation", "heating"]),
    remainder="passthrough",
    verbose_feature_names_out=False,
)
tree = make_pipeline(tree_prep, DecisionTreeRegressor(max_depth=2, random_state=0))
tree.fit(train[features], train["monthly_bill"])
print(export_text(tree[-1], feature_names=list(tree[0].get_feature_names_out())))
```

```text
|--- size_sqft <= 1955.00
|   |--- insulation_poor <= 0.50
|   |   |--- value: [149.07]
|   |--- insulation_poor >  0.50
|   |   |--- value: [182.41]
|--- size_sqft >  1955.00
|   |--- insulation_poor <= 0.50
|   |   |--- value: [185.75]
|   |--- insulation_poor >  0.50
|   |   |--- value: [219.04]
```

The first question is the one that cut the training error the most: is the home 1,955 square feet or smaller? Then, on both sides: is the insulation poor? (`insulation_poor <= 0.50` means "not poor".) A small, well-insulated home is guessed at $149.07; a large, poorly insulated one at $219.04. What the splits don't show: how much a feature matters to a deep tree with hundreds of questions, or whether a feature it never asks about matters. The year built never appears here, because insulation, which moves with it, was asked first. And a small change in the data can grow a quite different tree (*Regression Trees*).

## What a coefficient is not

The −7.92 doesn't mean a newer house lowers a bill, and $39.85 doesn't promise that new insulation saves $39.85 a month. Each is what this model learned from these 560 homes: what went together, with the other columns held the same. Correlation is not causation, and a model can only find correlations.

## Check yourself

2 questions to try on the lesson page.

## So what

A linear model's coefficients and a small tree's questions can be read directly, but only with care: on one scale, with the features that move together in mind, and never as causes.

## Sources

- scikit-learn, [Common pitfalls in the interpretation of coefficients of linear models](https://scikit-learn.org/stable/auto_examples/inspection/plot_linear_model_coefficient_interpretation.html), example: a coefficient is the relationship with the target assuming the other features remain constant; coefficients in different units can't be compared by size, and scaling the features (or multiplying each coefficient by its feature's standard deviation) puts them in one unit; when two features vary together their effect is hard to tease apart, and their coefficients vary and can swap sign; linear models measure association, not causation.
- scikit-learn, [Linear Models: Ordinary Least Squares](https://scikit-learn.org/stable/modules/linear_model.html#ordinary-least-squares), User Guide: when features are correlated, the estimates become highly sensitive to random errors (multicollinearity).
- scikit-learn 1.9.1, [OneHotEncoder](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html), API reference: `categories` lists the categories expected in each column; `drop='first'` drops the first category of each feature, which helps when perfectly collinear columns cause problems, such as in an unregularized linear regression.
- scikit-learn, [Decision Trees](https://scikit-learn.org/stable/modules/tree.html), User Guide: trees are simple to understand and to interpret; they can be unstable, because small variations in the data might result in a completely different tree.
- scikit-learn 1.9.1, [export_text](https://scikit-learn.org/stable/modules/generated/sklearn.tree.export_text.html), API reference: builds a text report showing the rules of a decision tree, with `feature_names` naming the features.

## Key terms

- [Coefficient](https://www.ask-codey.com/glossary/coefficient)
- [Standardization](https://www.ask-codey.com/glossary/standardization)
- [Multicollinearity](https://www.ask-codey.com/glossary/multicollinearity)
- [Decision Tree](https://www.ask-codey.com/glossary/decision-tree)

## Related

- [More Than One Feature](https://www.ask-codey.com/lessons/ml-several-features)
- [Why Explain a Model](https://www.ask-codey.com/lessons/ml-why-explain-a-model)
- [Permutation Importance](https://www.ask-codey.com/lessons/ml-permutation-importance)
- [Exercise: Explain the Energy Bill](https://www.ask-codey.com/lessons/exercise-ml-explain-the-energy-bill)

Source: https://www.ask-codey.com/lessons/ml-reading-coefficients-and-trees
