# Exercise: Delivery Time Errors

Write the mean absolute error, the root mean squared error, R squared and a check against the always-the-average baseline in plain Python for Pine Ridge Delivery's time estimates, and let pytest compare each one with scikit-learn's own function.

Journeyman · AI Basics · Python · Updated November 19, 2026

By Gregor Dzierzon

Pine Ridge Delivery's app tells every customer how long their order will take, and the team wants a report card for the model behind it: how far off it usually is, whether a few orders are badly off, and whether it beats saying the average to everyone. You'll write the four functions behind that report card in plain Python, the way *Errors and the Mean Absolute Error*, *Squared Errors and RMSE* and *R Squared: Better Than Guessing the Average?* worked them out by hand. Tests compare each one with scikit-learn's own function on the same values, so you'll know your arithmetic matches the library's to the ninth decimal place.

## Get the exercise code

Open this lesson's GitHub repository, [github.com/ask-codey/exercise-ml-delivery-time-errors](https://github.com/ask-codey/exercise-ml-delivery-time-errors), also linked from the **Exercise** box beside the lesson. Clone it into your `python-course` folder. In VS Code's terminal:

```bash
cd ~/python-course
git clone https://github.com/ask-codey/exercise-ml-delivery-time-errors.git
```

Git makes a folder, `exercise-ml-delivery-time-errors`, with the exercise code inside. Open that folder in VS Code (**File > Open Folder**), open its terminal with **Terminal > New Terminal**, and make the exercise's virtual environment. On Windows (PowerShell):

```powershell
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txt
```

On a Mac (zsh):

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

The prompt starts with `(.venv)` once the virtual environment is on. The exercise's README says what each command does, and what to do if PowerShell won't run the second one.

The folder holds:

- `deliveries.csv`: the 500 orders, our own made-up data, the same file the chapter's lessons use.
- `seed_data.py`: the script that made the CSV, with the pattern it planted, traffic jams included. You don't need to run it: the committed CSV is what keeps everyone's numbers the same.
- `delivery_errors.py`: the four functions, for you to finish.
- `explore.ipynb`: a notebook for looking at the orders, drawing the model's residuals, and trying your functions beside scikit-learn's. Choose the `.venv` kernel with **Select Kernel** (top right) before you run it. The tests don't read it.
- `report.py`: trains a model on 400 orders, guesses the other 100, and scores the guesses with your four functions. Ready to use.
- `tests/`: the tests, which call each function and check what comes back, and `conftest.py`, which gives a test a fresh copy of the orders. You don't need to change them.
- `requirements.txt` and `pytest.ini`: the packages the exercise needs, at exact versions, and where pytest finds the tests and your code.

## What to do

Every function in `delivery_errors.py` is already declared with its name and parameters, and for now it stops with a `NotImplementedError` that says *Not built yet.* The docstring under each one says what it should do, with examples:

```python
def rmse(actual, predicted):
    """Return the root mean squared error: the square root of the average squared error.

    Square each error (actual value minus guess), average the squares (that's
    the mean squared error, in squared minutes), then take the square root to
    get back to minutes. math.sqrt(x), or x ** 0.5, is the square root of x.

    rmse([28, 32, 35, 20, 48], [30, 30, 33, 18, 40]) -> 4.0   (squares 4, 4, 4, 4, 64; 80 / 5 = 16)
    rmse([30, 30], [20, 40])                         -> 10.0  (not 100.0, the squared minutes)
    """
    raise NotImplementedError("Not built yet.")
```

In each function, replace the `raise NotImplementedError("Not built yet.")` line with your own code, in plain Python: no scikit-learn, NumPy or pandas inside the functions (Python's own `math` is fine), and no rounding.

1. **`mae(actual, predicted)`** returns the mean absolute error: the size of each actual value minus its guess, averaged. `mae([30, 25, 40, 20], [24, 25, 30, 28])` returns `6.0`, and one order off by 6, `mae([30], [36])`, returns `6.0`.
2. **`rmse(actual, predicted)`** returns the root mean squared error: each error squared, averaged, then the square root. `rmse([28, 32, 35, 20, 48], [30, 30, 33, 18, 40])` returns `4.0`, and `rmse([30, 30], [20, 40])` returns `10.0`, minutes, not squared minutes.
3. **`r_squared(actual, predicted)`** returns 1 minus the model's squared errors added up, divided by the squared errors of always guessing the average of `actual`. `r_squared([10, 20, 30, 40], [12, 18, 33, 37])` returns `0.948`; perfect guesses give `1.0`, always guessing 25 gives `0.0`, and the guesses backwards, `[40, 30, 20, 10]`, give `-3.0`. The tests always give `actual` at least two different values.
4. **`beats_baseline(y_train, y_test, y_pred)`** returns `True` when the model's MAE on the test values is smaller than the MAE of guessing the average of `y_train` for every test value, as `DummyRegressor(strategy="mean")` does, and `False` otherwise, a tie included. `beats_baseline([20, 30, 40], [25, 35], [26, 34])` returns `True`: the model is off by 1.0 on average and the baseline, guessing 30, by 5.0.

Every function also works on a pandas column and a NumPy array, as `report.py` gives them: a `for` loop over `zip`, `len` and `sum` work the same on all three.

## Check your work

Run the tests from the exercise folder, with the virtual environment on:

```bash
python -m pytest
```

At first all 20 tests fail. That's expected: each function you finish turns its tests green. A failing test's message says what it called, what it expected and what came back, for example *rmse([30, 30], [20, 40]) should return 10.0, but it returned 100.0.* Several tests compare your answer with scikit-learn's `mean_absolute_error`, `root_mean_squared_error` or `r2_score` on the same eight orders, to 9 decimal places, and one compares `beats_baseline` with a real `DummyRegressor` on the 100 test orders. No test checks how good the model is: they check your arithmetic.

To see your functions at work on the whole file, run `python report.py`.

## Hints

- **One error at a time.** `zip(actual, predicted)` gives each actual value and its guess together in a `for` loop, as the Do Its in *Errors and the Mean Absolute Error* and *Squared Errors and RMSE* did.
- **Name the steps.** RMSE is the steps of its name from last to first: the errors, squared, their mean, then the root.
- **Two totals for R squared.** *R Squared: Better Than Guessing the Average?* adds up the model's squared errors and the average guess's squared errors in the same loop; the average is `sum(actual) / len(actual)`.
- **Reuse your own work.** `beats_baseline` can build a list of the training average, one per test value, and call your `mae` twice.
- **Which average?** The baseline guesses the *training* labels' average, the only one a model could know before the test.

## When you're done

Every test is green: pytest's last line says `20 passed`. And `python report.py` prints:

```text
1) orders: 500
2) training orders: 400
3) test orders: 100
4) MAE: 3.63
5) RMSE: 5.72
6) R squared: 0.69
7) baseline MAE: 8.26
8) beats the baseline: True
```

The same numbers the chapter's lessons got from scikit-learn, now from your own code. Look at lines 4 and 5: what does the gap between them say about the orders? In the notebook, find the orders whose residuals are over 10 minutes and look at their rows: does any column (the distance, the stops, the rain, the hour) set them apart, or are they the traffic jams `seed_data.py` describes, which no column shows?

## Submit your exercise

The exercise is how you finish this lesson. When your work is done, put it on GitHub in a repo of your own, then submit that repo's address here. *Hand In Your Work with Git* walks through each step.

1. On GitHub, click **+**, then **New repository**. Name it (for example `exercise-ml-delivery-time-errors`), choose **Public**, and click **Create repository**. Leave out the README: your folder already has one.
2. In VS Code's terminal, in your exercise folder, send your work to your new repo. Point your copy at your own repo, then commit and push:

   ```bash
   git remote set-url origin https://github.com/your-name/exercise-ml-delivery-time-errors.git
   git add .
   git commit -m "My solution"
   git push -u origin main
   ```

3. Paste your repo's address, such as `https://github.com/your-name/exercise-ml-delivery-time-errors`, into **Submit your exercise** on this page. The site checks that the repo is public and holds your commits, and then the lesson is complete.

Changed something after you submitted? Push it, and submit again: your latest submission is the one that counts.

## So what

Every error number a library prints is a few lines of arithmetic you can now write and check yourself, which is what lets you explain a model's report card, and trust it.

## Key terms

- [Mean Absolute Error](https://www.ask-codey.com/glossary/mean-absolute-error)
- [Root Mean Squared Error](https://www.ask-codey.com/glossary/root-mean-squared-error)
- [R Squared](https://www.ask-codey.com/glossary/r-squared)
- [Baseline](https://www.ask-codey.com/glossary/baseline-ai)
- [pytest](https://www.ask-codey.com/glossary/pytest)

## Related

- [Errors and the Mean Absolute Error](https://www.ask-codey.com/lessons/ml-mean-absolute-error)
- [Squared Errors and RMSE](https://www.ask-codey.com/lessons/ml-root-mean-squared-error)
- [R Squared: Better Than Guessing the Average?](https://www.ask-codey.com/lessons/ml-r-squared)
- [Baselines and Residual Plots](https://www.ask-codey.com/lessons/ml-regression-baselines-and-residual-plots)

Source: https://www.ask-codey.com/lessons/exercise-ml-delivery-time-errors
