A Report Script That Finds the Newest Data
A report script runs with one command instead of a person running notebook cells, finds the newest week's file with sorted and glob (which works because the file names start with the year), stacks every week into one table with concat, and takes its folder as an option through a main(argv) function, so running it again next week builds next week's report.
Journeyman · Python · Updated
After this lesson you can
- Find the newest data file in a folder with sorted and glob, and name files with year-month-day dates so they sort in order
- Stack every week's file into one table with concat, with a column that says which week each row came from
- Write a report script with a main(argv) function that reads a folder option with argparse, and run it again on next week's file
Key takeaways
- sorted(folder.glob("week-*.csv")) lists the week files in name order, and [-1] is the last of them, the newest.
- Name files with dates written year-month-day, such as week-2026-10-26.csv. Then name order is date order; week-9-28-2026.csv sorts after week-10-5-2026.csv.
- A loop that loads each week and adds a week column with assign, then pd.concat, gives every week in one table, each row labeled with its week.
- main(argv) reads its options from the list it's given. Under if name == "main", main(sys.argv[1:]) passes it the words typed after the script's name.
- Run the same command next week and it finds next week's file by itself. Nothing in the script names a week.
A notebook needs a person: someone opens it, runs the cells and reads the answers. A report that's due every Monday is better as a script, a .py file that does the whole job with one command, with no cells to run and nothing to edit. That's how a weekly sales summary, a lab's nightly data check or a club's monthly numbers get made without anyone remembering the steps.
The example is the Greenway Food Truck. A file of its sales arrives each Monday in data/incoming/, named by the day its week starts: week-2026-09-07.csv to week-2026-10-26.csv so far. Each row is one sale (sold_at, location, item, quantity, price, payment). Its functions already live in analysis.py: load_week(path) loads a week and adds a total column (quantity times price), and summarize(week) returns the week's headline numbers.
Which file is newest?
A script can't be told "this week's file" by a person each Monday; it has to find it. Paths and Folders with pathlib shows glob: a folder's files that match a pattern, where * stands for any text. sorted puts them in name order:
from pathlib import Path
files = sorted(Path("data/incoming").glob("week-*.csv"))
for path in files:
print(path.name)
print("newest:", files[-1].name)
week-2026-09-07.csv
week-2026-09-14.csv
week-2026-09-21.csv
week-2026-09-28.csv
week-2026-10-05.csv
week-2026-10-12.csv
week-2026-10-19.csv
week-2026-10-26.csv
newest: week-2026-10-26.csv
files[-1], read "files, minus one", is the last item of the list: the newest week. That only works because of how the files are named. Names are sorted letter by letter, like words in a dictionary, so a date sorts by date only when it's written year, then month, then day, each with its leading zero (2026-09-07). Here's what happens with dates written the American way:
print(sorted(["week-9-28-2026.csv", "week-10-5-2026.csv", "week-10-12-2026.csv"]))
['week-10-12-2026.csv', 'week-10-5-2026.csv', 'week-9-28-2026.csv']
1 comes before 9, so October sorts before September, and the "newest" file would be the oldest. When you choose how files are named, put the year first. (Sorting by the time a file was last changed is no safer: copying or downloading a file changes that time.)
Every week in one table
Dana, the truck's owner, also wants to see how this week compares with the others. Stacking Tables with concat stacks tables that share their columns; here each week gets a week column first, so every row says where it came from. A path's stem is its file name without .csv, and removeprefix("week-") takes week- off the front:
import pandas as pd
import analysis
tables = []
for path in files:
week_date = path.stem.removeprefix("week-")
tables.append(analysis.load_week(path).assign(week=week_date))
weeks = pd.concat(tables, ignore_index=True)
weeks.groupby("week")["total"].sum()
week
2026-09-07 4653.5
2026-09-14 4764.0
2026-09-21 4428.5
2026-09-28 4788.5
2026-10-05 4512.0
2026-10-12 4428.0
2026-10-19 4638.0
2026-10-26 4455.0
Name: total, dtype: float64
A script with main(argv)
Here's report.py, beside analysis.py. The work goes in a function, main, and the last two lines run it:
import argparse
import sys
from pathlib import Path
import pandas as pd
import analysis
def newest_week(folder):
files = sorted(Path(folder).glob("week-*.csv"))
if not files:
raise FileNotFoundError(f"No week-*.csv file in {folder}")
return files[-1]
def load_all_weeks(folder):
tables = []
for path in sorted(Path(folder).glob("week-*.csv")):
week_date = path.stem.removeprefix("week-")
tables.append(analysis.load_week(path).assign(week=week_date))
return pd.concat(tables, ignore_index=True)
def main(argv):
parser = argparse.ArgumentParser(description="Build the food truck's weekly report.")
parser.add_argument("--data", default="data/incoming", help="the folder of week files")
args = parser.parse_args(argv)
newest = newest_week(args.data)
week = analysis.load_week(newest)
weeks = load_all_weeks(args.data)
print("newest week:", newest.name)
for name, value in analysis.summarize(week).items():
print(f"{name}: {value}")
print("weeks so far:", weeks["week"].nunique())
if __name__ == "__main__":
main(sys.argv[1:])
Command-Line Options with argparse teaches the parser: --data is an option with a default, so the script works with no options at all and still lets you point it at another folder. Two details are particular to this script:
main(argv)takes the list of words to read. When you run the script, the last line passessys.argv[1:], the words typed afterreport.py. A notebook or a test can callmain(["--data", "data/old"])with its own list, without a terminal.- An empty folder stops with a clear message. With no files,
files[-1]would fail with anIndexErrorthat says nothing about folders;newest_weeksays what's wrong instead.
Run it, then run it again next week
From the terminal, in the folder that holds report.py:
python report.py
newest week: week-2026-10-26.csv
sales: 4455.0
items_sold: 537
best_item: Street Tacos
weeks so far: 8
Next Monday, a new file arrives: week-2026-11-02.csv. You run the very same command, and the script finds it:
newest week: week-2026-11-02.csv
sales: 4777.5
items_sold: 591
best_item: Street Tacos
weeks so far: 9
Nothing in report.py named a week. With --data pointing at a folder with no week files, python report.py --data nowhere ends with its own message:
FileNotFoundError: No week-*.csv file in nowhere
The script still prints to the terminal. Writing the Report: Charts and Tables on a Web Page turns its output into a page Dana can open.
Check yourself
2 questions to try on the lesson page.
So what
A script that finds its own newest file and takes its folders as options turns "update the report" into one command, the same one every week.