Explaining a Model

People affected by a model's guess deserve to know what drives it. Read a linear model's coefficients and a tree's questions with care, measure which features a model leans on with permutation importance, see how a guess changes as one feature changes, and write it all in words a homeowner can follow, including what the model can't tell you. The running example is a utility estimating homes' monthly energy bills.

Updated

Why Explain

  1. Why Explain a Model (Journeyman): A model's guess lands on people who ask why, the person who gets the estimate, the team that keeps the model working and the manager who decides whether to use it, so you explain either the whole model (what it leans on across every example) or one guess (why this home got this number), and every explanation describes what the model learned from its data, not why the world works that way.

Looking Inside a Model

  1. Reading Coefficients and Trees (Journeyman): A linear regression trained on standardized features gives coefficients you can compare, each the change in the guess for one standard deviation more of its feature with the others held the same, but features that move together share and swap credit, a small tree's top splits show the questions it asks first, and neither a coefficient nor a split is a cause.
  2. Permutation Importance (Freelancer): To see which features any fitted model leans on, shuffle one column of held-out data so its link to the label is broken, score the model again, and take the difference, which scikit-learn's permutation_importance repeats n_repeats times with a random_state, giving a fairer answer than a tree's built-in feature_importances_, though features that move together can each look less important than they are.
  3. How a Guess Changes with One Feature (Freelancer): Changing one feature of a home while keeping the rest the same, and watching the model's guess, gives that home's individual line, and averaging every home's line gives the partial dependence, which PartialDependenceDisplay.from_estimator draws with kind='both', but it shows how the model behaves, not cause and effect, and it can't be trusted where the data is thin.

Explaining to a Person

  1. Explaining a Result to Someone Who Isn't Technical (Journeyman): A plain-language explanation of a model's result says what it predicts, how far off it usually is in the reader's own units, the few things it leans on most, what it doesn't know and where it shouldn't be trusted, in words the reader uses, without the jargon of a too-technical draft, the emptiness of a too-vague one, or the false precision of a guess given to the cent.

Exercise: Explain the Energy Bill

  1. Exercise: Explain the Energy Bill (Freelancer): Write three functions that measure which features Greenway Home Energy's random forest leans on with permutation importance on its test homes, pick the few that matter most, and turn them into four plain sentences for a homeowner, and let pytest check the call's settings, the sorting and its ties, and every sentence.