# Exercise: Model Check-Up

Give Pine Ridge Delivery's year-old model its check-up in Python, saving the whole pipeline with joblib and loading it back only when the scikit-learn version matches, reporting each column's share of new orders outside the training range and its change in mean, and building a model card that refuses an empty section, with pytest checking each step.

Journeyman · AI Basics · Python · Updated November 19, 2026

By Gregor Dzierzon

A year after Pine Ridge Delivery's app started using its model, the team wants a check-up they can run any time: save the model safely and load it back, see whether this year's orders still look like the ones it learned from, and keep a model card that says what it's for and where it must not be used. You'll write the four functions behind that check-up, the way *When the World Changes*, *Saving and Loading a Model* and *Model Cards* worked them out. Tests check each one, so you'll know the saved model comes back the same and your drift numbers are exact.

## Get the exercise code

Open this lesson's GitHub repository, [github.com/ask-codey/exercise-ml-model-check-up](https://github.com/ask-codey/exercise-ml-model-check-up), also linked from the **Exercise** box beside the lesson. Clone it into your `python-course` folder. In VS Code's terminal:

```bash
cd ~/python-course
git clone https://github.com/ask-codey/exercise-ml-model-check-up.git
```

Git makes a folder, `exercise-ml-model-check-up`, with the exercise code inside. Open that folder in VS Code (**File > Open Folder**), open its terminal with **Terminal > New Terminal**, and make the exercise's virtual environment. On Windows (PowerShell):

```powershell
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txt
```

On a Mac (zsh):

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

The prompt starts with `(.venv)` once the virtual environment is on. The exercise's README says what each command does, and what to do if PowerShell won't run the second one.

The folder holds:

- `deliveries.csv` and `deliveries_new.csv`: last year's 500 orders and this year's 300, our own made-up data, the same files the chapter's lessons use.
- `seed_data.py`: the script that made both files, with what changed in a year. You don't need to run it: the committed CSVs are what keep everyone's numbers the same.
- `check_up.py`: the four functions, for you to finish, and `SECTIONS`, the card's seven sections.
- `explore.ipynb`: a notebook for comparing the two years' orders, drawing their distances, and trying your functions. Choose the `.venv` kernel with **Select Kernel** (top right) before you run it. The tests don't read it.
- `report.py`: trains the model, saves and loads it, checks this year's orders and builds the card with your functions. Ready to use.
- `tests/`: the tests, which call each function and check what comes back, and `conftest.py`, which gives a test fresh copies of the orders and the trained model. You don't need to change them.
- `requirements.txt` and `pytest.ini`: the packages the exercise needs, at exact versions, and where pytest finds the tests and your code.

## What to do

Every function in `check_up.py` is already declared with its name and parameters, and for now it stops with a `NotImplementedError` that says *Not built yet.* The docstring under each one says what it should do, with examples:

```python
def load_model(path):
    """Load a file that save_model wrote, and return the pipeline inside it.

    Read the dictionary with joblib.load. If its "sklearn_version" isn't the version
    running now (sklearn.__version__), don't return the model: raise a ValueError whose
    message names both versions, in this shape:
        Saved with scikit-learn 1.8.0, but this is scikit-learn 1.9.1.
    Only ever load files you made yourself: loading a joblib file can run code inside it.

    model = load_model("delivery_model.joblib")  -> the pipeline, ready to predict
    """
    raise NotImplementedError("Not built yet.")
```

In each function, replace the `raise NotImplementedError("Not built yet.")` line with your own code.

1. **`save_model(pipeline, path)`** saves one dictionary to `path` with `joblib.dump`: `"model"` holds the whole pipeline, scaler and all, and `"sklearn_version"` holds `sklearn.__version__`, the version running now. It returns nothing.
2. **`load_model(path)`** reads that dictionary back with `joblib.load` and returns the pipeline. If the file's `"sklearn_version"` isn't the version running now, it raises a `ValueError` with the message *Saved with scikit-learn 1.8.0, but this is scikit-learn 1.9.1.*, with the file's version first and the running version second.
3. **`drift_report(train, new, columns)`** returns a dictionary with one entry per column in `columns`, in that order and no others. Each entry is a dictionary with `"share_outside"`, the share of `new`'s values below `train`'s smallest or above its largest (a value equal to either is inside), and `"mean_change"`, `new`'s mean minus `train`'s. With training distances `[1.0, 2.0, 4.0, 5.0]` and new ones `[2.0, 6.0, 7.0, 5.0]`, the distance entry is `{"share_outside": 0.5, "mean_change": 2.0}`: 6.0 and 7.0 are above 5.0, 2 of 4, and the mean went from 3.0 to 5.0. It doesn't change either table.
4. **`model_card(model, date_trained, intended_use, not_for, data, performance, limits)`** returns a dictionary whose keys are the seven `SECTIONS`, in order, `"Model"` holding `model`, `"Date trained"` holding `date_trained`, and so on. It raises a `ValueError` whose message starts with the section's name when `model`, `date_trained`, `intended_use` or `data` is empty or only spaces, when `not_for` or `limits` is an empty list, or when `performance` doesn't have an `"overall"` entry and at least one group beside it.

## Check your work

Run the tests from the exercise folder, with the virtual environment on:

```bash
python -m pytest
```

At first all 19 tests fail. That's expected: each function you finish turns its tests green. A failing test's message says what it called, what it expected and what came back. The save-and-load tests write their files in `tmp_path`, a temporary folder pytest makes for each test and removes later, so they never touch your exercise folder; they check the loaded pipeline makes exactly the same guesses for all 100 test orders, that it's the whole pipeline, and that a file saved by another version is refused. The drift tests use small tables written in the test file, so every number can be worked out by hand, and then the two CSV files. No test checks how good the model is.

To see your functions at work on the whole files, run `python report.py`.

## Hints

- **One dictionary in the file.** `save_model` and `load_model` agree on the dictionary's two keys; `load_model` compares `"sklearn_version"` before it returns anything.
- **The range comes from the training rows.** `train[column].min()` and `train[column].max()` are the edges; `new[column] < low` gives one `True` or `False` per row, and `.sum()` counts the `True` ones.
- **Strictly outside.** A value equal to the smallest or the largest is inside, so use `<` and `>`, not `<=` and `>=`.
- **Build the card, then check it.** `" ".strip()` is `""`, which counts as empty; `len(not_for) == 0` finds an empty list; `"overall" in performance` checks for the overall entry.
- **Stop with a message.** `raise ValueError(f"{section} is empty.")` stops the function there and shows the message after the error's type; each docstring gives its message's exact words.

## When you're done

Every test is green: pytest's last line says `19 passed`. And `python report.py` prints:

```text
1) same guesses after loading: True
2) distance_miles: outside the training range 0.25, mean change +1.87
3) stops_before: outside the training range 0.00, mean change -0.05
4) is_raining: outside the training range 0.00, mean change +0.02
5) model card sections: Model, Date trained, Intended use, Not for, Data, Performance, Limits
6) MAE in minutes (overall, dry, raining): 3.63 3.91 2.33
```

The drift report says the distances moved and the stops didn't. But *When the World Changes* found the stops cost more time this year. Line 3 can't show that: what can? In the notebook, run the cell that averages this year's errors by the number of stops. Then write one more item for the card's *Limits*, in words a delivery manager would understand, as a Markdown cell at the end of the notebook. There's no test for that one: when the manager would know what to watch for, it works.

## Submit your exercise

The exercise is how you finish this lesson. When your work is done, put it on GitHub in a repo of your own, then submit that repo's address here. *Hand In Your Work with Git* walks through each step.

1. On GitHub, click **+**, then **New repository**. Name it (for example `exercise-ml-model-check-up`), choose **Public**, and click **Create repository**. Leave out the README: your folder already has one.
2. In VS Code's terminal, in your exercise folder, send your work to your new repo. Point your copy at your own repo, then commit and push:

   ```bash
   git remote set-url origin https://github.com/your-name/exercise-ml-model-check-up.git
   git add .
   git commit -m "My solution"
   git push -u origin main
   ```

3. Paste your repo's address, such as `https://github.com/your-name/exercise-ml-model-check-up`, into **Submit your exercise** on this page. The site checks that the repo is public and holds your commits, and then the lesson is complete.

Changed something after you submitted? Push it, and submit again: your latest submission is the one that counts.

## So what

A model you can save safely, check against new data and describe honestly is a model other people can rely on, and know when not to.

## Key terms

- [Data Drift](https://www.ask-codey.com/glossary/data-drift)
- [Model Card](https://www.ask-codey.com/glossary/model-card)
- [joblib](https://www.ask-codey.com/glossary/joblib)
- [Model Persistence](https://www.ask-codey.com/glossary/model-persistence)
- [pytest](https://www.ask-codey.com/glossary/pytest)

## Related

- [When the World Changes](https://www.ask-codey.com/lessons/ml-when-the-world-changes)
- [Saving and Loading a Model](https://www.ask-codey.com/lessons/ml-saving-a-model)
- [Model Cards](https://www.ask-codey.com/lessons/ml-model-cards)

Source: https://www.ask-codey.com/lessons/exercise-ml-model-check-up
