Sign in to open this module
All 24 modules are open from the start — nothing here is locked, and nothing costs anything. Sign in so your progress, titles and credentials stay with you, on every device you use.
Free forever, with your Google account. No password, no payment.
Checking your account…
Capstone: Your First Sprint
A real engineering job, in fifteen programs
No lesson in this module and no new syntax — you already have all of it. What you have not had is the job: a messy daily export, colleagues who describe what they want rather than which function to call, and an on-call engineer who will not schedule anything that cannot tell him whether it worked. Fourteen tickets land across the sprint. Friday, the tool ships.
You have joined Kestrel Supply Co
Read this once before you start. It is the whole of the handover you are going to get, which is also true of the real thing.
Kestrel Supply Co
A distributor with three sales regions. Every morning the billing system drops a CSV export, and every morning a human being opens it in a spreadsheet, builds a pivot table, and pastes three numbers into Slack. It takes twenty minutes and it has been wrong twice.
Engineer — first sprint
Your first piece of work is the thing everybody has wanted for a
year and nobody has had time to build: report, a
command-line tool that does the morning job in one command.
In cron by Friday
The tickets arrive in the order they were sent. The last one is the tool itself, reviewed by your lead and scheduled by your SRE.
Who is asking
Four people, and none of them will tell you which function to reach for. They will tell you what the tool has to do, and — more usefully — what went wrong the last time something did not. Those complaints are the requirements.
Bea Cho · Finance analyst
Does the job by hand today. Knows every quirk in the export, because she has hit all of them.
Yvonne Adeyemi · Head of RevOps
Posts the number every morning. Wants it ranked, wants it configurable, and reads the top two.
Ravi Menon · SRE
Will schedule it. Cares about exit codes, streams, and error messages that name the file.
Tomas Lindqvist · Engineering lead
Reviews the code. Cares that it can be imported, called and tested without a shell.
The export
One file, six rows, dropped by the billing system every morning. Every ticket that needs data writes this same file in its starter, so the numbers stay the same all week and you will recognise them by Wednesday.
| region | quantity | price |
|---|---|---|
| eu | 3 | 1250 |
| us | 4 | 300 |
| eu | 2 | 500 |
| jp | 1 | 900 |
| us | 6 | 500 |
| eu | x | 100 |
Revenue is quantity × price, so the five usable rows
come to 9850 — eu 4750, us 4200, jp 900. The last row
is not a mistake in the file: billing writes x while an
order is mid-correction, it will never be fixed, and it is the row every
naive version of this tool crashes on.
Handover notes
A CSV is all text. csv.DictReader hands
you "3", not 3. Adding those together
concatenates them rather than failing, which is the worst of both
outcomes.
A bad row is not an emergency. Crash on it and one mid-correction stops the morning. Skip it silently and the totals quietly shrink. Skip it, count it, and report the count.
Nothing here reads your output. Cron reads an exit code; a pipe reads stdout. A tool that prints "failed" and exits 0 is indistinguishable from one that worked.
The file it writes is shared. The ranking lands where the dashboard reads it, so "do everything except the writing" has to be real — checked before the file is opened, not after.
The habit worth carrying into every ticket
Write functions that take what they need and hand back what they produced. A function that reads a file, prints a table and exits the process can only be tested by running the whole program and reading the screen. One that takes rows and returns pairs can be called with three made-up rows and checked in a line — which is exactly what the checks below do, and exactly what Tomas means every time he says "make it testable".
Real Python runs right here in your browser — nothing to install, nothing sent to a server. The interpreter downloads once the first time you press Run, then stays cached.
To do
Bea Cho · Finance analyst
"Welcome aboard. So you know what you're replacing: every morning I
download sales.csv from billing, open it in a
spreadsheet, and the first thing I do is check the row count against
yesterday's, because twice now the export has come through truncated
and nobody noticed until the numbers were already in Slack."
Start where Bea starts. The starter writes today's export to disk for you — the same six rows every ticket this week uses — and your job is to read it back and say how big it is.
A CSV's first line is the header, so the number of records is one fewer than the number of lines. Work both out from the file rather than typing them: an export that arrives truncated is exactly the case where a hardcoded number lies.
Deliver: two lines.
7 lines
6 data rowswith open("sales.csv") as f: lines = f.read().splitlines(). Then print(f"{len(lines)} lines") and print(f"{len(lines) - 1} data rows").
To do
Bea Cho · Finance analyst
"Row count's a start, but I need the columns. Can you print each row out so I can eyeball it? Region, quantity, price, one row per line."
csv.DictReader reads the header for you and hands back one
dictionary per row, keyed by column name. Splitting on commas yourself
works right up until a field contains one.
Everything a CSV gives you is text
row["quantity"] is the string "3", not the
number 3. Adding those up concatenates them. Converting is your job,
and tomorrow's ticket is about what happens when a value will not
convert.
Deliver: the header line as it appears in the file, then one line per row.
region,quantity,price
eu 3 1250
us 4 300
eu 2 500
jp 1 900
us 6 500
eu x 100reader.fieldnames is the list of column names — ",".join(reader.fieldnames) gives the header line back. Then for row in reader: print(row["region"], row["quantity"], row["price"]).
To do
Bea Cho · Finance analyst
"You've seen the x in the quantity column. Billing does
that occasionally when an order is mid-correction — it's not an error
on our side and it'll never be fixed. Whatever you build has to skip
those and keep going, and it has to tell me how many it
skipped, because if that number jumps I want to know."
Two decisions live in this ticket, and the second is the one that matters. Crashing on a bad row means one mid-correction stops the whole morning. Skipping it silently means the totals quietly shrink and nobody finds out. Skip it, count it, and say so.
int("x") raises ValueError. Catch that
specifically — a bare except would also swallow the typo in
your own code.
Deliver: the usable rows converted to numbers, and the two counts.
loaded 5
rejected 1Inside the loop, a try block doing row["quantity"] = int(row["quantity"]) and the same for price, then good.append(row). In `except ValueError:` do rejected += 1 and continue.
To do
Yvonne Adeyemi · Head of RevOps
"The number I actually post every morning is revenue by region. Quantity times price, added up per region. Right now Bea does it with a pivot table and I take her word for it."
Accumulate into a dictionary keyed by region.
defaultdict(int) saves the "have I seen this region before"
branch: a key that has never been touched starts at 0, so
+= works the first time.
Print them sorted by region name, so two runs of the same export produce the same output — a report whose row order moves around is a report people stop trusting.
Deliver: one line per region, alphabetical.
eu 4750
jp 900
us 4200for row in good: summed[row["region"]] += row["quantity"] * row["price"]. Then for region in sorted(summed): print(region, summed[region]).
To do
Yvonne Adeyemi · Head of RevOps
"Alphabetical is no good for the standup — I want biggest first, and honestly I only read the top two. Can you make the count a setting though? Some mornings I want all of them."
Put the ranking in a function of its own. rank takes the
totals and returns pairs, largest first — no printing, no files. That
separation is what lets a test call it with rows it invented, and it is
the shape the graded assignment is built around.
sorted takes a key saying what to sort on and
reverse=True for descending. Write the key as a named
function rather than a lambda if it makes the intent clearer — either
is fine, and one of them has a name.
Deliver: rank(summed) returning
(region, total) pairs largest first, then the top two
printed.
eu 4750
us 4200def by_total(pair): return pair[1] — then return sorted(totals.items(), key=by_total, reverse=True). A dictionary's .items() gives exactly the pairs you want to sort.
To do
Tomas Lindqvist · Engineering lead
"Nice work on the totals. Now stop editing the file to change what it reads — the day someone changes the wrong line at 8am, we ship yesterday's numbers. It takes the path as an argument, and it takes the region filter as an option. That's the difference between a script and a tool."
A script edited before each run is not a tool. The first thing to move out is whatever you keep changing at the top of the file.
parser = argparse.ArgumentParser(prog="report")
parser.add_argument("path")
args = parser.parse_args(argv)
A positional argument is required and unnamed — the
thing the command is about. parse_args takes a list, which
is what lets you test it without a shell.
Your task: build a parser taking a path
positional and a --region option defaulting to
all, then print the two attributes for two different
argument lists:
sales.csv all
sales.csv euparser.add_argument("path") for the positional, and parser.add_argument("--region", default="all") for the option. The names on the parsed object come from the argument names.
To do
Yvonne Adeyemi · Head of RevOps
"Two things. I want to choose how many rows I see, and I want to pick the output format — csv or json, nothing else. And if I typo one of those I want to be told straight away, not get a stack trace three seconds in."
type=int converts and rejects.
choices restricts and lists the valid values in the error.
Both happen in the parser, so the rest of the program can assume the
arguments are sane.
parser.add_argument("--top", type=int, default=3)
parser.add_argument("--format", choices=["csv", "json"], default="csv")
A bad value makes argparse print the usage and exit with code 2 — the conventional code for "you called this wrongly".
Your task: add both, then print the defaults, a converted value with its type, and the exit code from a bad one:
3 csv
5 int
json
exit 2add_argument("--top", type=int, default=3) and add_argument("--format", choices=["csv", "json"], default="csv"). Nothing else changes — argparse does the converting and the rejecting.
To do
Ravi Menon · SRE
"Before this goes anywhere near cron it needs a switch that makes it do everything except the writing, so I can test the schedule without clobbering the shared file. And a verbose switch for when it misbehaves at 6am."
action="store_true" makes a flag: False by
default, True when the option appears. No value follows it.
Watch the name. --dry-run arrives as
args.dry_run, because a hyphen cannot be part of an
attribute name.
Your task: add --dry-run and
--verbose, then print both for three argument lists:
False False
True False
True Trueadd_argument("--dry-run", action="store_true") and the same for --verbose. The attribute for the first one is dry_run, with an underscore.
To do
Tomas Lindqvist · Engineering lead
"One structural note and then I'll leave you alone. Put the work in a function that takes the arguments and hands back an exit code. If the only way to run it is from a shell, the only way to test it is from a shell, and nobody will."
The shape that makes a tool testable is one function taking the arguments and returning the exit code.
def main(argv=None):
args = parser.parse_args(argv)
...
return 0
Taking argv is what lets a check call
main(["sales.csv", "--top", "2"]) without a shell, and
passing None makes argparse fall back to the real command
line — so the normal path is unaffected.
Your task: write main(argv=None) printing a
summary line and returning 0. Call it twice:
reading sales.csv, top 3
0
reading other.csv, top 5
0args = parser.parse_args(argv), then print(f"reading {args.path}, top {args.top}"), then return 0. The print of the return value happens at the call site, which is why each call produces two lines.
To do
Ravi Menon · SRE
"Cron doesn't read your output. It reads one number. If the export comes through empty and this thing exits 0, the pipeline marks it green and we find out on Monday. Give me the conventional codes."
The convention is worth following, because other programs depend on it:
Success
It did what it was asked.
Failure
It ran, and something was wrong — a missing file, no usable rows.
Usage error
You called it wrongly. argparse already does this one for you.
A shell script with set -e, a cron job and a CI step all
check the code and none of them read the message. Without one, a failure
is indistinguishable from a success.
Your task: write main(argv) returning
0 when there are rows to report and 1 when the
file is empty. Print a line either way:
3 rows
0
no rows to report
1A guard: if not rows, print the complaint and return 1. Otherwise print the count and return 0. Early return keeps both paths at one level of indentation.
To do
Ravi Menon · SRE
"I want to pipe this straight into the alerting job. That only works if the output is the answer and nothing else — no counts, no progress notes, no 'loaded 5 rows'. Send that stuff where it belongs."
Results go to stdout; diagnostics go to
stderr. The reason is piping:
report sales.csv | mail -s totals team should send the
totals, not "rejected 2 rows".
print(row)
print(f"rejected {n} rows", file=sys.stderr)
Both reach a terminal, so a person running it still sees everything. The difference only shows when the output is redirected — which is exactly when it matters.
Your task: print the two data rows to stdout and the two diagnostics to stderr. Only the data will appear in the output below, which is the point:
eu 4750
us 1200A loop printing f"{region} {total}" for the rows, then two prints with file=sys.stderr for the diagnostics.
To do
Ravi Menon · SRE
"Billing missed an upload last month and the tool we had said 'error'. That was the whole message. I spent forty minutes finding out which file it meant. Whatever this prints when it can't read something, it says the path."
FileNotFoundError already carries the path. Catching it and
printing something vaguer is a downgrade — name the file, say what you
were doing, put it on stderr, and return a failing code.
Your task: write main(argv) that reads the
file and reports its line count, or reports the missing path on stderr
and returns 1:
2 lines
0
1
The missing run prints nothing to stdout — its complaint went to stderr — so only its exit code appears.
Wrap the open in try. In `except FileNotFoundError as error:` print f"cannot read {args.path}: {error.strerror}" to stderr and return 1. On success print the line count to stdout and return 0.
To do
Yvonne Adeyemi · Head of RevOps
"The ranking file it writes is the one the dashboard reads. So the dry run has to be a genuine dry run — I want to see what it would have written, with the file untouched. If I can't trust that, I'll never run it on a Monday."
A dry run reads, validates, and reports what it would do — then stops before changing anything. On a tool that overwrites a file it is the difference between one people will run and one they will not.
if args.dry_run:
print(f"would write {len(rows)} rows to {args.out}")
else:
write(args.out, rows)
Your task: write main(argv) honouring
--dry-run. The real run writes the file; the dry run says
what it would have written and leaves the file alone:
would write 2 rows to out.txt
False
wrote 2 rows to out.txt
True
The booleans are whether the file exists after each run.
An if on args.dry_run: print the "would write" line and return 0 before touching anything. Otherwise open args.out with "w", write the rows one per line, and print the "wrote" line.
To do
Tomas Lindqvist · Engineering lead
"Last thing before you ship. I want to import your module from the nightly job and call main() directly — which I can't do if importing it runs the whole report and then kills my process. The exit goes at the very edge, not inside."
sys.exit belongs in the __main__ guard, not
inside main. A function that exits the process cannot be
called from a test, or reused by another tool.
if __name__ == "__main__":
sys.exit(main())
The guard also means importing the file runs nothing — the point Modules & the Standard Library made, and the reason one file can be both a tool and a library.
Your task: move the sys.exit out of
main and into a guard at the bottom:
2
0
still running
2
The lab runs your code as the main module, so the guard fires and
calls main a second time — which is the fourth line, and a
useful thing to watch happen. Its sys.exit(0) then ends the
program, which is why nothing follows it.
Change sys.exit(0) to return 0, then add `if __name__ == "__main__":` with sys.exit(main()) indented under it at the bottom of the file.
Fri 11:00 — ship the tool
To doTomas Lindqvist · Engineering lead
"Everything's in the tickets. Put it in one file and I'll review it — and then Ravi puts it in cron on Monday, so it had better behave when nobody is watching it. Bea stops doing this by hand on Tuesday."
The week, assembled. Build report: a command-line tool that
reads the sales export, totals it by region, prints the top few, and
optionally writes the whole ranking to a file. Every piece of it is a
ticket you have already done — the work is putting them in one file
without any of them breaking the others.
The interface. Build a parser named report
with:
path— positional, the CSV to read--top— a whole number, default2--out— a path to write the full ranking to, defaultNone--dry-run— a flag
Write totals(rows): takes a list of
DictReader dictionaries and returns
(region, total) pairs, largest first. It must not open a file
and must not print.
Write main(argv=None), which:
- reads the path with
csv.DictReader, skipping rows whose numbers will not convert - prints the top
--toprows to stdout, asregion total - prints
loaded N, rejected Mto stderr - with
--outand no--dry-run, writes the full ranking there and printswrote N rows to <path>to stdout - with
--outand--dry-run, printswould write N rows to <path>instead, and creates nothing - returns
0normally,1when the file is missing or no rows survived — printing the reason to stderr in both cases
Finish with the __main__ guard calling
sys.exit(main()).
Print exactly eight lines — three runs, each followed by its exit code:
eu 4750
us 4200
0
eu 4750
us 4200
would write 3 rows to out.csv
0
1
The runs are: a plain run showing the top two; the same run with
--out and --dry-run, which still shows the top two
and then says what it would have written; and a run against a file that does
not exist, which prints nothing to stdout because its complaint went to
stderr.
The parser takes "path", "--top" with type=int and default 2, "--out" with default None, and "--dry-run" with action="store_true". totals accumulates into a defaultdict(int) and returns sorted(..., key=..., reverse=True) with a named key function. In main: wrap the open in try/except FileNotFoundError, printing to stderr and returning 1. Loop the DictReader with a try round the two int() calls, counting rejects. If nothing survived, complain to stderr and return 1. Print the top rows with a slice, then the loaded/rejected line to stderr. Then, only when --out was given, either the "would write" line or the real write followed by "wrote". Return 0.