◀ Course contents Part 3 · Module 3-06

Reading & Writing Files

Where the data was before it was in your program

Everything so far arrived as a string somebody typed into the source. Real data lives in files, and this is the last piece of Part 3: opening one safely, reading it without loading a gigabyte into memory, and writing one back without destroying what was already there. The lab runs a real filesystem, so every file you write here genuinely exists until the page is reloaded.

Ready?

`open()`, and Why `with` Is Not Optional

open() gives you a file object. The second argument is the mode, and it decides what you are allowed to do and what happens to whatever was there before:

"r"   read; the file must exist         (the default)
"w"   write; CREATES OR EMPTIES the file
"a"   append; creates, and adds to the end
"x"   create; fails if it already exists

An open file is a resource the operating system is holding for you, and it has to be given back. A with block does that automatically, at the end of the block, whether it finished normally or raised:

with open("notes.txt", "w") as f:
    f.write("first line\n")
# closed here, even if the write raised

Without it you must call f.close() yourself, and any exception between the open and the close skips it. Data sitting in a buffer that never got flushed is a file that looks empty for no visible reason — which is why with is not a style preference.

`write()` does not add a newline

print() spoiled you. f.write("a") followed by f.write("b") produces ab, on one line. Every line you write needs its \n writing too.

Quick check

What does open("report.txt", "w") do to an existing report?

Three Ways to Read, and When Each Is Right

with open("log.txt") as f:
    everything = f.read()        # one string, whole file in memory

with open("log.txt") as f:
    lines = f.readlines()        # a list of lines, newlines still attached

with open("log.txt") as f:
    for line in f:               # one line at a time, nothing else held
        ...

The third is the one to reach for by default. It reads a line, hands it over, and forgets it — so a four-gigabyte log costs the same memory as a four-line one. The first two are fine when you know the file is small and you need all of it at once.

Every line you read this way still has its newline on the end. Comparing line == "END" against "END\n" fails, which is why almost every loop over a file starts with line = line.strip().

Reading a file that is not there raises FileNotFoundError — which is worth catching only when a missing file is a real possibility you have an answer for. A missing config with a sensible default, yes. A missing input file the whole job depends on, no: let it stop.

Quick check

Why iterate a file rather than call read()?

Writing Without Destroying Anything

The distinction that matters most in this whole module:

Empties it first

`open(path, "w")`

The truncation happens at the open, before a single write.

Keeps what is there

`open(path, "a")`

Every write goes on the end. This is what a log wants.

pathlib handles the common cases without any of this ceremony, and handles paths properly on every operating system:

from pathlib import Path

p = Path("data") / "log.txt"     # joins correctly, everywhere
p.exists()
p.suffix                          # ".txt"
p.write_text("one\ntwo\n")       # opens, writes, closes
lines = p.read_text().splitlines()
p.parent.mkdir(parents=True, exist_ok=True)

write_text and read_text are whole-file operations, so the memory caveat applies — but for a config, a small report or a JSON document they replace four lines with one, and write_text uses mode "w", so the same truncation rule applies.

One more thing worth knowing before it bites: text files have an encoding. The default depends on the machine, which means a file that reads fine on your laptop can raise UnicodeDecodeError on a server. Passing encoding="utf-8" explicitly removes an entire class of deployment surprise.

Quick check

A daily job opens its report with mode "w", then crashes before writing. What is in the file?

Read, Transform, Write

Almost every file job is the same three steps, and keeping them separate is what makes the middle one testable:

def clean(lines):              # no files in here at all
    return [l.strip() for l in lines if l.strip()]

raw = Path("in.txt").read_text().splitlines()
Path("out.txt").write_text("\n".join(clean(raw)) + "\n")

clean takes a list and returns a list. It can be tested in one line, with no file anywhere near it — which is exactly the argument from module 3-01, applied to the messiest part of real work.

A function that opens a file, transforms its contents and writes another cannot be tested without a filesystem, and every test of it is slower and more fragile than it needed to be.

Read at the edge

One place that knows about paths.

Transform in the middle

Lists and dictionaries in, lists and dictionaries out.

Write at the other edge

One place that knows the output format.

Never in the middle

A transform that also writes is two jobs, and neither can be checked on its own.

This lab has a real filesystem

Files you write here genuinely exist, in memory, inside the tab. They survive between runs of the same exercise — so an exercise that appends will keep appending if you press Run twice, and one that starts by writing its input fresh will not. Reloading the page wipes everything.

Nothing you do here can reach a file on your computer. The sandbox has no route to it.

Quick check

Why split reading, transforming and writing into separate functions?

0 of 9 completed

Real Python runs right here in your browser — nothing to install, nothing sent to a server. The interpreter downloads once the first time you press Run, then stays cached.

To do

open() takes a path and a mode. "w" writes, "r" reads, and a with block closes the file at the end whether or not anything went wrong.

with open("notes.txt", "w") as f:
    f.write("first\n")

write() adds no newline of its own — print() spoiled you. Every line you write needs its \n writing too.

Your task: write three lines to notes.txt, then read the whole file back and print it, then print its length in characters:

alpha
beta
gamma
17

your_code.py

# Write three lines, then read the file back.
contents =

print(contents.rstrip())
print(len(contents))

Python

Hint

Two with blocks: one opening in "w" to write "alpha\n", "beta\n" and "gamma\n", one opening in "r" to read. contents = f.read() in the second.

Output

To do

This code opens a file, raises before reaching close(), and leaves the file open with its data still sitting in a buffer. The file exists and is empty, for no reason visible in the file itself.

A with block closes the file when the block ends — normally or by exception. That is the whole reason it exists, and why calling close() by hand is a thing you should almost never do.

Your task: rewrite it with a with block, so that even though the calculation still fails, the two lines already written are safely on disk:

could not finish: division by zero
['alpha', 'beta']

your_code.py

f = open("partial.txt", "w")
f.write("alpha\n")
f.write("beta\n")

try:
    total = 10 / 0
    f.write("gamma\n")
except ZeroDivisionError as error:
    print("could not finish:", error)

f.close()

with open("partial.txt") as check:
    print(check.read().splitlines())

Python

Hint

Put the open in a with block and indent the writes and the try inside it. Then the close line goes away entirely — the block does it.

Output

To do

Looping over a file hands you one line per pass and holds nothing else, so a four-gigabyte log costs the same memory as a four-line one.

Every line arrives with its newline still attached, which is why almost every loop over a file starts by stripping it. Comparing line == "END" against "END\n" fails, silently and confusingly.

Your task: read queue.txt line by line, skipping blanks and comments, and print each remaining job numbered:

1. build
2. test
3. deploy
Jobs: 3

your_code.py

with open("queue.txt", "w") as f:
    f.write("build\n\n# skipped\ntest\ndeploy\n")

jobs = []
# Read the file line by line and collect the real jobs.

for number, job in enumerate(jobs, start=1):
    print(f"{number}. {job}")
print("Jobs:", len(jobs))

Python

Hint

with open("queue.txt") as f: then for line in f: — strip each line, continue past the empty ones and the ones starting with #, and append the rest.

Output

To do

This is the most expensive mistake in the module. "w" empties the file at the moment it is opened, before anything is written. Run a job twice with "w" inside the loop and only the last entry survives.

"w"   empties, then writes
"a"   keeps everything, adds to the end

A log wants "a". A report generated fresh each time wants "w". Choosing is a decision, not a default.

Your task: run it and watch the log come out with one line in it. Then fix the mode so all three entries survive:

['boot', 'retry', 'done']
3

your_code.py

with open("audit.txt", "w") as f:
    f.write("")

for entry in ["boot", "retry", "done"]:
    with open("audit.txt", "w") as f:
        f.write(entry + "\n")

with open("audit.txt") as f:
    lines = f.read().splitlines()

print(lines)
print(len(lines))

Python

Hint

The mode inside the loop is the problem. Leave the first block alone — that one is deliberately starting the log empty — and change the one in the loop to "a".

Output

To do

Most file reading is not to print the file — it is to work something out from it. The reading is the boring part; the counting is the job.

Your task: read the log and report on it: how many lines, how many are errors, and the longest line's length. Skip blank lines entirely.

Lines: 4
Errors: 2
Longest: 18

your_code.py

with open("service.log", "w") as f:
    f.write("INFO boot complete\n\nERROR db timeout\nINFO retry\nERROR db timeout\n")

total = 0
errors = 0
longest = 0

# One pass over the file.

print("Lines:", total)
print("Errors:", errors)
print("Longest:", longest)

Python

Hint

Loop the file, strip each line, continue past the blanks. Then add one to total, add one to errors when the line starts with "ERROR", and keep the longest with an if comparing len(line) to longest.

Output

To do

pathlib replaces four lines with one for the common cases, and joins paths correctly on every operating system:

from pathlib import Path

p = Path("data") / "notes.txt"   # joins properly, everywhere
p.write_text("one\ntwo\n")
p.read_text().splitlines()
p.exists()
p.suffix                          # ".txt"

write_text uses mode "w", so the same truncation rule applies — it is shorter, not safer.

Your task: build the path data/notes.txt, make its directory, write two lines, then print the lines, whether it exists, its suffix, and its name:

['one', 'two']
True
.txt
notes.txt

your_code.py

from pathlib import Path

p =

# Make the directory, write the file, then four lines.

Python

Hint

Path("data") / "notes.txt" builds the path. p.parent.mkdir(parents=True, exist_ok=True) makes the folder without complaining if it is already there. Then write_text, and read_text().splitlines() for the first line of output.

Output

To do

Opening a file that does not exist raises FileNotFoundError. Catch it only when a missing file is a real possibility you have an answer for.

A missing config with sensible defaults, yes. A missing input file the whole job depends on, no — let it stop, loudly, with the path in the message.

Your task: write load_config(path), which returns the file's lines, or an empty list if the file is not there. Print the result for a file that exists and one that does not:

['port=8080', 'debug=false']
[]
2
0

your_code.py

with open("app.conf", "w") as f:
    f.write("port=8080\ndebug=false\n")

def load_config(path):
    # The lines, or an empty list when there is no such file.
    pass

print(load_config("app.conf"))
print(load_config("nope.conf"))
print(len(load_config("app.conf")))
print(len(load_config("nope.conf")))

Python

Hint

try: open the path and return f.read().splitlines(). except FileNotFoundError: return []. An empty list is a real answer here — the caller can loop over it without checking.

Output

To do

A comma-separated file is the most common thing you will be handed. Read it, split each line, convert what needs converting, and it becomes the lists and dictionaries from Part 2.

The first line is usually a header, and it is not data — skipping it is the step everybody forgets exactly once.

Your task: read stock.csv, skip the header, and build records as a list of dictionaries. Then print the count, one record, and the total value:

3
{'name': 'tea', 'quantity': 3, 'price': 1250}
17880

your_code.py

with open("stock.csv", "w") as f:
    f.write("name,quantity,price\ntea,3,1250\nwhisk,12,990\ncloth,5,450\n")

records = []
# Read the file, skip the header, build one dictionary per row.

print(len(records))
print(records[0])
print(sum(r["quantity"] * r["price"] for r in records))

Python

Hint

Read the lines, then walk them from index 1 onwards to skip the header — or use enumerate and continue on the first. Split each row into three, convert the two numbers, and append a dictionary with the keys name, quantity and price.

Output

To do

Almost every file job is read, transform, write. Keeping the middle step in a function that takes a list and returns a list is what makes it testable — no files anywhere near it.

def clean(lines):
    return [l.strip() for l in lines if l.strip()]

A function that opens a file, transforms it and writes another cannot be checked without a filesystem, and every test of it is slower and more fragile than it needed to be.

Your task: write clean(lines) — which takes a list and returns a list, dropping blanks and comments and upper-casing the rest — then use it to turn in.txt into out.txt:

['BUILD', 'TEST', 'DEPLOY']
3

clean must not open, read or write anything.

your_code.py

with open("in.txt", "w") as f:
    f.write("build\n\n# skip me\ntest\ndeploy\n")

def clean(lines):
    # A list in, a list out. No files.
    pass

# Read in.txt, clean it, write out.txt.

with open("out.txt") as f:
    result = f.read().splitlines()

print(result)
print(len(result))

Python

Hint

clean is one comprehension: keep the stripped line when it is non-empty and does not start with #, and upper-case it. Then read in.txt into a list, pass it through clean, and write "\n".join(...) + "\n" to out.txt.

Output

The log rotation

To do

Every night a service's log is rotated: the day's entries are cleaned into an archive file, a summary is written alongside it as JSON, and the rest of the pipeline reads those two rather than the raw log. Write that job.

The raw log has blank lines, comment lines from the rotation tool, and trailing whitespace. A real entry is timestamp LEVEL message, padded so the levels line up.

Write these two, with exactly these names:

parse(lines) — takes a list of raw lines and returns a list of (level, message) tuples, dropping blanks and comments. It must not open, read or write anything.
summarise(entries) — takes that list and returns a dictionary with total, levels (a plain dict of counts) and first_error (the message of the first ERROR, or None). Also no files.

Then run the job: read service.log, write every cleaned entry to archive.log as LEVEL message, one per line, and write the summary to summary.json with sorted keys. Then read both files back and report.

Print exactly five lines:

Archived: 5 entries
INFO: 2
WARN: 1
ERROR: 2
First error: db timeout after 30s

The level lines are in the order INFO, WARN, ERROR — not alphabetical, and not whatever order they happened to appear in.

your_code.py

import json

with open("service.log", "w") as f:
    f.write(
        "2026-09-02 INFO   boot complete\n"
        "2026-09-02 WARN   slow query 1.4s  \n"
        "\n"
        "2026-09-02 ERROR  db timeout after 30s\n"
        "# rotated by logctl\n"
        "2026-09-02 INFO   retry scheduled\n"
        "2026-09-02 ERROR  db timeout after 30s\n"
    )

def parse(lines):
    # A list of raw lines in, a list of (level, message) tuples out. No files.
    pass

def summarise(entries):
    # A list of tuples in, a summary dict out. No files.
    pass

# Read the log, write archive.log and summary.json, then read both back.

# Five lines of output.

Python

Hint

parse: strip each line, skip the empty ones and the ones starting with #, split with maxsplit=2 into stamp, level and message, and collect (level, message) — remembering to strip the message, since the log is padded. summarise: total is len(entries); levels is a counting loop or a Counter converted with dict(); first_error is the message of the first entry whose level is ERROR, and None when there is none. Then write the archive with "\n".join(f"{level} {message}" for ...) and json.dump the summary. For the level lines, loop over ["INFO", "WARN", "ERROR"] rather than over the dictionary.

Output

Sign in to open this module

Reading & Writing Files

open(), and Why with Is Not Optional

write() does not add a newline

Quick check

Three Ways to Read, and When Each Is Right

Quick check

Writing Without Destroying Anything

open(path, "w")

open(path, "a")

Quick check

Read, Transform, Write

Read at the edge

Transform in the middle

Write at the other edge

Never in the middle

This lab has a real filesystem

Quick check

Out to a file and back

The close that never ran

One line at a time

The mode that empties the file

Reading a file to answer a question

Paths without the ceremony

When the file is not there

A file of records

The three steps, kept apart

The log rotation

Notification

`open()`, and Why `with` Is Not Optional

`write()` does not add a newline

`open(path, "w")`

`open(path, "a")`