# Up Next: Answering Questions

Look back at what Module 2 taught, finding and handling missing values, fixing types, removing duplicates, tidying text and dates, catching impossible values and putting it all in a cleaning function, try a cleaning log of your own, and see what Module 3 adds, filtering, sorting, grouping, pivot tables, joins and reshaping.

Apprentice · Python · Updated November 19, 2026

By Gregor Dzierzon

## After this lesson you can

- Describe what filtering, sorting, grouping, joining and reshaping will let you answer with clean data

## Key takeaways

- After Module 2 you can turn a messy file into a clean one, honestly, and make it again with one function call.
- Every cleaning step is a decision, and a short log of what each step changed and why lets someone else trust the result.
- Clean data is where analysis starts. Module 3 asks it questions, such as which month was coldest, by filtering, sorting, grouping, joining and reshaping.

When this module started, a messy file could fool you: a blank read as zero, `$75` as text, a row counted twice, -999 °F in an average. Now you can find each of those, decide what to do about it, and write the decision down as code that runs the same way every time. That's where much of an analyst's working day goes, and now you can do it.

This lesson looks back at what you can do, gives you something to try with all of it, and shows what Module 3 adds: questions answered with the clean data.

## What you can do now

After Module 2, you can:

- **find missing values**, the hidden ones too, and drop or fill them honestly, with a flag column for every filled value;
- **fix types**, turning text such as `59 lbs` and `$75` into numbers without losing a value silently;
- **remove duplicates**, telling a double entry from two real visits;
- **tidy text and dates**, so spellings match and dates are real dates;
- **catch values that can't be right**, turning sentinels into `NaN`, flagging impossible readings, and keeping the real outliers;
- **check your data** with `assert`, every time it's cleaned;
- **put every step in one function** that leaves the raw table alone, and **save the clean file** beside the raw one, never over it.

## Try it: a cleaning log

A clean file is easier to trust when it comes with a note that says what was done to it. Try writing one for the Ridgeview School weather station's readings, from *Exercise: Weather Station Cleanup*, with only what you know now:

- **One line for each step** of the cleaning, in the order it runs.
- **What the step changed,** as a count worked out with pandas, such as how many values became `NaN` or how many rows were dropped.
- **Why,** in plain words: "the station writes -999 when its sensor drops out".
- **The rows before and after,** at the top and the bottom.

Write it as a Markdown cell at the end of the exercise's notebook, or as a few `print` lines in a script. There's no test for this one: when someone who has never seen the station could read your log and know what happened to the data, it works.

## What Module 3 unlocks

Clean data is where the questions start. Module 3 answers them:

- **Selecting, filtering and sorting:** pick the rows and columns that matter, such as every reading during a storm, and sort them into a top ten.
- **Grouping and summarizing:** `groupby` splits a table into groups and summarizes each one, such as the average temperature of each month; pivot tables lay out two groupings at once, and `resample` totals by week or month.
- **Joining and reshaping:** combine two tables on a shared column, catch a join that goes wrong, and turn a wide table into a long one and back.

Here's a sneak peek, on the clean file the exercise saves. `parse_dates=` reads the timestamps as dates, as *Parsing Dates with to_datetime* explains, and `groupby` does the rest:

```python
import pandas as pd

clean = pd.read_csv("data/clean/readings_clean.csv", parse_dates=["timestamp"])
months = clean["timestamp"].dt.month_name()
clean.groupby(months, sort=False)["temp_f"].mean().round(1)
```

```text
timestamp
September    63.0
October      47.7
November     42.2
December     31.6
January      27.8
February     30.3
March        39.9
April        46.5
May          61.6
June         69.0
Name: temp_f, dtype: float64
```

A school year's temperatures by month, in three lines, from data you cleaned yourself. You don't need to understand every part of it yet: that's what Module 3 is for.

## Check yourself

2 questions to try on the lesson page.

## So what

Module 2 turned messy files into clean ones you can trust and rebuild. Module 3 puts that clean data to work, answering the questions people actually ask.

## Key terms

- [DataFrame](https://www.ask-codey.com/glossary/dataframe)
- [groupby()](https://www.ask-codey.com/glossary/groupby-python)

## Related

- [A Cleaning Function, Step by Step](https://www.ask-codey.com/lessons/pandas-cleaning-function)
- [Grouping with groupby](https://www.ask-codey.com/lessons/pandas-groupby)

Source: https://www.ask-codey.com/lessons/data-up-next-answering-questions
