# Splits for Time and Groups

Shuffled folds assume every row is independent of the others, which isn't true for data in time order, where shuffling lets a model learn from the future, or for data with repeats, where the same customer lands on both sides; TimeSeriesSplit always tests on rows that come after the ones it learned from, and GroupKFold keeps each group on one side.

Freelancer · AI Basics · Python · Updated November 19, 2026

By Gregor Dzierzon

## After this lesson you can

- Explain why shuffled folds overrate a model on data in time order, and score it with TimeSeriesSplit instead
- Keep every row of a group, such as one member's visits, on the same side of each split with GroupKFold

## Key takeaways

- Shuffled folds let a model learn from showings that came after the ones it's tested on. On data in time order, that makes the score look better than a real forecast will do.
- TimeSeriesSplit learns from the earliest rows and tests on the rows right after them, with a bigger past each time. It needs the rows in time order.
- On Lakeside Cinema's showings, the same tree scored 0.72 on shuffled folds and 0.59 when it had to guess the future.
- When one customer has several rows, GroupKFold keeps all of them on one side of every split, so the model is tested on customers it never saw.

Lakeside Cinema doesn't need to guess the tickets for a showing that already happened in March. It needs next month's. So the honest test for its model is the one it will face in use: learn from the past, guess the future. Shuffled folds don't test that. Neither do they test a model of members' habits on members it has never met. You'll meet both kinds of data constantly, sales by date, readings over time, many rows per customer, patient or student, and each needs its own way to split.

The example is Lakeside Cinema's `showings.csv`, from *Exercise: Honest Scores*. Each code block is one notebook cell, run in order: the first one loads the data, and the others use what it made.

## What a shuffle hides

The showings are in date order, from March to December, and the crowds grow slowly over the months:

```python
import pandas as pd
from sklearn.model_selection import GroupKFold, KFold, TimeSeriesSplit, cross_val_score
from sklearn.tree import DecisionTreeRegressor

showings = pd.read_csv("showings.csv")
features = ["day_number", "weekday", "start_hour", "is_new_release", "screen_seats"]
X = showings[features]
y = showings["tickets_sold"]
print("First showing:", showings["date"].iloc[0], "| last showing:", showings["date"].iloc[-1])
print("Average tickets, first 150 showings:", round(y.iloc[:150].mean(), 1))
print("Average tickets, last 150 showings:", round(y.iloc[-150:].mean(), 1))
```

```text
First showing: 2025-03-03 | last showing: 2025-12-27
Average tickets, first 150 showings: 113.3
Average tickets, last 150 showings: 139.3
```

Shuffle these showings into folds, and each fold's training rows include showings from after the ones it's tested on: a showing on a Tuesday in June is guessed by a tree that has seen the Monday and Wednesday around it, and December too. In use, the model never gets that look at the future. The score it earns that way is inflated, a kind of **data leakage**: something the model couldn't know at prediction time slipped into training.

## TimeSeriesSplit: past, then future

`TimeSeriesSplit` cuts the rows in their order and always tests on the rows right after the ones it learned from. Each split learns from a longer past:

```python
for number, (fit_rows, check_rows) in enumerate(TimeSeriesSplit(n_splits=5).split(X), start=1):
    print("Split", number, "| learns from", showings["date"].iloc[fit_rows[0]], "to", showings["date"].iloc[fit_rows[-1]],
          "|", len(fit_rows), "showings | tested on", showings["date"].iloc[check_rows[0]], "to", showings["date"].iloc[check_rows[-1]])
```

```text
Split 1 | learns from 2025-03-03 to 2025-04-21 | 150 showings | tested on 2025-04-22 to 2025-06-10
Split 2 | learns from 2025-03-03 to 2025-06-10 | 300 showings | tested on 2025-06-11 to 2025-07-30
Split 3 | learns from 2025-03-03 to 2025-07-30 | 450 showings | tested on 2025-07-31 to 2025-09-18
Split 4 | learns from 2025-03-03 to 2025-09-18 | 600 showings | tested on 2025-09-19 to 2025-11-07
Split 5 | learns from 2025-03-03 to 2025-11-07 | 750 showings | tested on 2025-11-08 to 2025-12-27
```

Every test block is 150 showings, 50 days, and comes after everything its tree learned from. It works by position, so the rows must be sorted by time first; these already are.

![Five rows, one per split, along a timeline from March to December 2025: in each row a training bar starts in March and grows longer, 150, 300, 450, 600 and 750 showings, and right after it a test block of 150 showings, so the test block moves later each time, ending with November 8 to December 27](https://fsdtrainingmedia.blob.core.windows.net/public/images/2026/11/ca578b1ffc644c9dbfe87f8a22016d33.webp)

Now the same depth-5 tree, scored both ways:

```python
tree = DecisionTreeRegressor(max_depth=5, random_state=0)
shuffled = cross_val_score(tree, X, y, cv=KFold(n_splits=5, shuffle=True, random_state=0))
in_order = cross_val_score(tree, X, y, cv=TimeSeriesSplit(n_splits=5))
print("Shuffled folds:", shuffled.round(2), "| mean", round(shuffled.mean(), 2))
print("Past, then future:", in_order.round(2), "| mean", round(in_order.mean(), 2))
```

```text
Shuffled folds: [0.72 0.67 0.73 0.73 0.74] | mean 0.72
Past, then future: [0.45 0.51 0.67 0.65 0.65] | mean 0.59
```

0.72 against 0.59. The shuffled tree could learn the slow rise from showings on both sides of each test day; the tree that only knows the past has to guess crowds bigger than most it has seen, and a tree's guesses never go beyond the averages of its leaves. The earliest splits score worst: 150 showings is a short past. If the cinema will use the model to guess next month, 0.59 is the number to expect. In a real project, the newest showings would also be held back as the final test set.

## The same member on both sides

The second trap is repeats. Lakeside Cinema's members' club records each visit, and some members come often. A model of how much a member spends on snacks would learn each regular's habits. Test it on other visits by the same members, and it looks better than it will for a member who joins next week. Here are eight visits by four members:

```python
visits = pd.DataFrame({
    "member_id": ["M-01", "M-01", "M-01", "M-02", "M-02", "M-03", "M-03", "M-04"],
    "visits_this_month": [6, 5, 7, 1, 2, 3, 3, 4],
    "snacks_spent": [31, 28, 35, 6, 9, 15, 14, 22],
})
by_visit = KFold(n_splits=2, shuffle=True, random_state=0).split(visits)
by_member = GroupKFold(n_splits=2).split(visits, groups=visits["member_id"])
for name, splits in [("KFold", by_visit), ("GroupKFold", by_member)]:
    for fit_rows, check_rows in splits:
        learned = set(visits["member_id"].iloc[fit_rows])
        tested = set(visits["member_id"].iloc[check_rows])
        print(name, "| tested on", sorted(tested), "| on both sides:", sorted(learned & tested))
```

```text
KFold | tested on ['M-01', 'M-03', 'M-04'] | on both sides: ['M-01', 'M-03']
KFold | tested on ['M-01', 'M-02', 'M-03'] | on both sides: ['M-01', 'M-03']
GroupKFold | tested on ['M-01', 'M-04'] | on both sides: []
GroupKFold | tested on ['M-02', 'M-03'] | on both sides: []
```

`KFold` splits visits with no idea who made them, so M-01 and M-03 are learned from and tested on at once. `GroupKFold` takes each visit's group, here `member_id`, and keeps every visit of a member in the same fold, so each member is tested on as someone the model never met. Pass the same `groups` to `cross_val_score(..., groups=..., cv=GroupKFold(...))` to score that way. *Leaky Features and Duplicate Rows*, in the *Data Leakage* chapter, comes back to repeats with real scores.

## Which split fits

- **Rows independent of each other:** shuffled `KFold`, seeded.
- **Rows in time order, and the model will guess the future:** `TimeSeriesSplit`, on rows sorted by time.
- **Several rows per customer, member, patient or store:** `GroupKFold`, with that ID as `groups`.

## Check yourself

2 questions to try on the lesson page.

## So what

Test a model the way it will be used: on the future for a forecast and on new people for a model of people, because a shuffle that mixes them in can make a model look better than it will ever be.

## Sources

- scikit-learn, [Cross-validation: evaluating estimator performance](https://scikit-learn.org/stable/modules/cross_validation.html), User Guide: independent and identically distributed data is a common assumption that rarely holds in practice; for samples from a time-dependent process a time-series aware scheme is safer, and for data with a group structure a group-wise one; shuffling data in time order can give an overfit model and an inflated validation score; `GroupKFold` ensures the same group is not in both the testing and training sets.
- scikit-learn 1.9.1, [TimeSeriesSplit](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html), API reference: for time-ordered data, where other cross-validation methods would train on future data and evaluate on past data; the k-th split trains on the first k folds and tests on the next one, successive training sets are supersets of the earlier ones, and each test set is `n_samples // (n_splits + 1)` rows by default.
- scikit-learn 1.9.1, [GroupKFold](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GroupKFold.html), API reference: a k-fold variant with non-overlapping groups, where each group appears in the test set exactly once across the folds; the groups are passed to `split` as `groups`.
- scikit-learn 1.9.1, [KFold](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html), API reference: `KFold` is not affected by groups (its `split` ignores `groups`), and `shuffle` with an integer `random_state` gives the same folds on every call.

## Key terms

- [Time Series](https://www.ask-codey.com/glossary/time-series)
- [Cross-Validation](https://www.ask-codey.com/glossary/cross-validation)
- [Data Leakage](https://www.ask-codey.com/glossary/data-leakage)

## Related

- [Cross-Validation](https://www.ask-codey.com/lessons/ml-cross-validation)
- [Validation Sets: Keep the Test Set for Last](https://www.ask-codey.com/lessons/ml-validation-and-test-sets)
- [Exercise: Honest Scores](https://www.ask-codey.com/lessons/exercise-ml-honest-scores)

Source: https://www.ask-codey.com/lessons/ml-splits-for-time-and-groups
