# Probabilities and the 0.5 Line

predict_proba gives each example one probability per class, in the order of classes\_, predict picks the class with the highest one, which for two classes is a line at 0.5, and the probability is the model's confidence, not a guarantee, so you can draw the line somewhere else when the job needs it.

Journeyman · AI Basics · Python · Updated November 19, 2026

By Gregor Dzierzon

## After this lesson you can

- Read predict_proba's columns in the order of classes\_, and pick out the positive class's probability
- Turn probabilities into labels at a threshold, and explain why a probability is confidence, not a guarantee

## Key takeaways

- predict_proba gives one column per class, in the order of classes_, and each row adds up to 1. With classes [0, 1], column 1 is the chance of spam.
- For two classes, predict picks the class whose probability is higher, which is the same as calling spam above 0.5.
- A probability is the model's confidence, learned from past examples, not a guarantee. Some emails at 0.88 aren't spam, and some at 0.1 are.
- You can draw the line somewhere else. A lower line flags more emails as spam, and a higher one flags fewer.

Inbox Guard doesn't have to treat every email the model calls spam the same way. An email it's nearly sure about can go straight to the spam folder; one it's unsure about could stay in the inbox with a warning. That choice needs more than a yes or a no: it needs the model's probability. Whenever a model's guess sets off an action (a reminder email, a fraud hold, a doctor's second look), you'll read its probabilities and decide where to draw the line.

The example is Inbox Guard's `emails.csv`, from *Exercise: Spam or Not*. Each code block is one notebook cell, run in order: the first one loads the data, splits it and trains the model from *Logistic Regression*.

## One column per class

`predict_proba` (read "predict proba", short for probability) gives each email a row of probabilities, one for each class:

```python
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

emails = pd.read_csv("emails.csv")
FEATURES = ["num_links", "num_exclamations", "mentions_free", "sender_in_contacts", "subject_all_caps"]
X_train, X_test, y_train, y_test = train_test_split(
    emails[FEATURES], emails["is_spam"], test_size=0.2, random_state=42
)
model = LogisticRegression()
model.fit(X_train, y_train)
probabilities = model.predict_proba(X_test)
print("Classes:", model.classes_)
print(probabilities[:4].round(3))
```

```text
Classes: [0 1]
[[0.118 0.882]
 [0.971 0.029]
 [0.641 0.359]
 [0.781 0.219]]
```

The columns come in the order of `classes_`: first the chance of class `0` (not spam), then of class `1` (spam). The first test email is 0.118 not spam and 0.882 spam. Each row adds up to 1, because the email has to be one or the other. To get the chance of spam for every email, take column 1:

```python
spam_chance = probabilities[:, 1]
print("Chances of spam:", spam_chance[:4].round(3))
print("Guesses:        ", model.predict(X_test.head(4)))
```

```text
Chances of spam: [0.882 0.029 0.359 0.219]
Guesses:         [1 0 0 0]
```

`probabilities[:, 1]` reads "every row, column 1". Take it by position from `classes_`, never by habit: with the classes `['not spam', 'spam']`, spam is column 1 too, but with `['spam', 'trusted']` it would be column 0.

## predict draws the line at 0.5

`predict` picks the class with the highest probability. With two classes, that's the same as calling an email spam when its chance is above 0.5:

```python
at_half = (spam_chance > 0.5).astype(int)
print("Same as predict for all 200:", (at_half == model.predict(X_test)).all())
```

```text
Same as predict for all 200: True
```

`(spam_chance > 0.5)` gives `True` or `False` for each email, and `.astype(int)` turns those into 1 and 0. An email at exactly 0.5 is called not spam by `predict`, which needs more than 0.5; real probabilities almost never land exactly on it. The line's name is the **threshold** (your textbook may say *decision threshold*).

## Confidence, not a guarantee

A probability is how sure the model is, from the patterns in the emails it learned from. It isn't a promise about one email:

```python
print("Chance of spam:", spam_chance[0].round(3))
print("Guess:", model.predict(X_test.head(1))[0])
print("What it really was:", y_test.iloc[0])
```

```text
Chance of spam: 0.882
Guess: 1
What it really was: 0
```

88.2% sure, and wrong. Across many emails, the probabilities do mean something:

```python
sure = spam_chance >= 0.8
unlikely = spam_chance < 0.2
print("Given 0.8 or more:", sure.sum(), "emails, and", y_test[sure].sum(), "were spam")
print("Given under 0.2:", unlikely.sum(), "emails, and", y_test[unlikely].sum(), "were spam")
```

```text
Given 0.8 or more: 7 emails, and 6 were spam
Given under 0.2: 107 emails, and 10 were spam
```

`y_test[sure]` keeps only the emails where `sure` is `True`, and adding up their 1s counts the spam. "Nearly sure" was right 6 times out of 7, and "unlikely" still let 10 spam emails through. A model that's right most of the time is never right every time, so a filter that acts on a probability needs a way back for the emails it gets wrong, such as a spam folder people can check.

## Drawing the line somewhere else

0.5 is the default, not a rule. You can count an email as spam when its chance is at or above any line you choose:

```python
for threshold in [0.3, 0.5, 0.7]:
    flagged = (spam_chance >= threshold).sum()
    print("Line at", threshold, "flags", flagged, "of 200 emails")
```

```text
Line at 0.3 flags 63 of 200 emails
Line at 0.5 flags 32 of 200 emails
Line at 0.7 flags 14 of 200 emails
```

A lower line catches more spam and moves more good emails to the spam folder; a higher line does the opposite. Which line is right depends on which mistake costs more, and choosing it fairly (from validation data, never the test emails) is what *Moving the Threshold* teaches.

## Do it: flag the likely spam

Inbox Guard's model has scored four new emails, and you want the ones to send to the spam folder.

The inputs:

- `rows`: what `predict_proba` gave, one pair per email: the chance it's not spam, then the chance it's spam (the classes are `[0, 1]`).
- `email_ids`: each email's ID, in the same order as `rows`.
- `chances`: each email's chance of spam, in the same order.
- `threshold`: the line. An email whose chance is at or above it is flagged.

For example:

```python
email_ids = ["IG-2001", "IG-2002", "IG-2003", "IG-2004"]
rows = [[0.9, 0.1], [0.25, 0.75], [0.5, 0.5], [0.6, 0.4]]
```

| Email | Row | Chance of spam | At or above 0.5? |
|---|---|---|---|
| IG-2001 | [0.9, 0.1] | 0.1 | no |
| IG-2002 | [0.25, 0.75] | 0.75 | yes |
| IG-2003 | [0.5, 0.5] | 0.5 | yes |
| IG-2004 | [0.6, 0.4] | 0.4 | no |

So `spam_chances(rows)` returns `[0.1, 0.75, 0.5, 0.4]`, and `flag_spam(email_ids, [0.1, 0.75, 0.5, 0.4], 0.5)` returns `["IG-2002", "IG-2003"]`.

**Your job:** write two functions.

- `spam_chances(rows)` returns a list of each row's chance of spam: the second number of each pair, in order.
- `flag_spam(email_ids, chances, threshold)` returns a list of the IDs whose chance is at or above `threshold`, in the order they're given.
- A chance exactly on the line is flagged.
- If no chance reaches the line, or the lists are empty, `flag_spam` returns `[]`.

Practiced live on the lesson page: write the Python right there and check it.

## Check yourself

2 questions to try on the lesson page.

## So what

A classifier's probabilities are more useful than its yes or no: read the positive class's column by `classes_`, treat each one as confidence rather than a promise, and draw the line where the job needs it.

## Sources

- scikit-learn 1.9.1, [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html), API reference: `predict_proba` returns an array of shape (n_samples, n_classes) with the probability of each class, ordered as in `classes_`; in the binary case, `decision_function` gives the score for `classes_[1]`, where a score above 0 means that class is predicted.
- scikit-learn, [Glossary of Common Terms and API Elements](https://scikit-learn.org/stable/glossary.html): *predict_proba* outputs one column per class in binary classification, and each row adds to 1; *classes_* gives the order of `predict_proba`'s columns.
- scikit-learn, [Linear Models: Logistic regression](https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression), User Guide: the predicted probability is used as a classifier by applying a threshold, by default 0.5.
- scikit-learn, [Tuning the decision threshold for class prediction](https://scikit-learn.org/stable/modules/classification_threshold.html), User Guide: for binary classification, `predict` uses a cut-off of 0.5 on the probability from `predict_proba`, which isn't the best cut-off for most uses.

## Key terms

- [predict_proba()](https://www.ask-codey.com/glossary/predict-proba)
- [Probability](https://www.ask-codey.com/glossary/probability)
- [Decision Threshold](https://www.ask-codey.com/glossary/decision-threshold)
- [Logistic Regression](https://www.ask-codey.com/glossary/logistic-regression)

## Related

- [Logistic Regression](https://www.ask-codey.com/lessons/ml-logistic-regression)
- [Stratified Splits](https://www.ask-codey.com/lessons/ml-stratified-splits)
- [Exercise: Spam or Not](https://www.ask-codey.com/lessons/exercise-ml-spam-or-not)

Source: https://www.ask-codey.com/lessons/ml-predicted-probabilities
