◀ Course contents Part 3 · Module 3-03

Comprehensions

The loop that builds a list, written as one line

Three quarters of the loops you have written have the same shape: start an empty list, walk something, append a transformed item, use the list. A comprehension is that shape with the scaffolding removed. It is not a trick and it is not about being short — it is about the line saying what you want rather than the four steps that produce it.

Ready?

Four Lines Become One

This loop appears constantly:

squares = []
for n in range(1, 6):
    squares.append(n * n)

And this is the same thing:

squares = [n * n for n in range(1, 6)]

Read it left to right: what to collect, then where the items come from. The for half is identical to the loop you would have written; the expression on the left is what used to go inside append().

The brackets decide the result. Square brackets build a list, curly braces with a colon build a dictionary, curly braces without one build a set, and round brackets build something lazier that the last lesson covers.

The loop variable does not leak

After the loop above, n still exists and holds 5. After the comprehension it does not exist at all — a comprehension gets its own scope. One fewer name left lying around, and one fewer accidental reuse.

Quick check

Which loop shape is a comprehension a direct replacement for?

Two Places an `if` Can Go, and They Mean Different Things

An if at the end filters. Items that fail it never reach the result:

[n for n in numbers if n % 2 == 0]     # only the even ones

An if/else at the front is the conditional expression from module 2-01. It chooses what to collect, and every item still appears:

["pass" if s >= 60 else "fail" for s in scores]

Filtering shortens the list; the conditional expression changes the values. Mixing them up is the most common comprehension mistake, and it produces a result of the wrong length rather than an error.

Both halves can do work, so a filter and a transformation live happily together:

[name.strip().title() for name in raw.split(",") if name.strip()]

The filter is evaluated first, so this never calls .title() on the blanks it is dropping.

Same length

`[x if x > 0 else 0 for x in nums]`

Every item survives; the negatives become zero.

Shorter

`[x for x in nums if x > 0]`

The negatives are gone entirely.

Quick check

A list of ten numbers, and you write [n for n in nums if n > 5]. How long is the result?

Dictionaries and Sets, Same Idea

Change the brackets and the punctuation, and the same shape builds the other containers.

# dict: a key and a value, separated by a colon
{name: len(name) for name in names}

# set: no colon, and duplicates disappear
{tag.strip().lower() for tag in raw.split(",")}

The set version is worth noticing: it deduplicates and cleans in one pass, where the loop would have been four lines and a seen check.

A dictionary comprehension is the usual way to reshape data that arrived in the wrong form — inverting a mapping, or turning a list of pairs into something you can look up:

by_sku = {row.split(",")[0]: row for row in rows}
inverted = {value: key for key, value in counts.items()}

Both of those have a catch worth knowing: keys are unique, so if two rows share a SKU the later one silently wins. A comprehension will not warn you, and neither will a loop — but the comprehension is short enough that the risk is easier to see.

Quick check

What is the difference between {x for x in items} and {x: x for x in items}?

Generators, and Knowing When to Stop

Round brackets build a generator expression: the same shape, but nothing is computed until something asks for it, and no list is ever built.

total = sum(int(x) for x in raw.split(","))

For a total, the list was never wanted — only the sum. On a few items that saves nothing; on a ten-million-line file it is the difference between working and running out of memory.

The catch is that a generator is consumed once. Sum it twice and the second answer is zero, because there is nothing left to walk. If you need the values more than once, build a list.

And the thing worth saying last: a comprehension is not always the right answer.

Two levels of nesting

[y for x in xs for y in x if ...] is a puzzle. The loop is longer and readable.

Side effects

A comprehension whose expression prints or appends elsewhere is a loop pretending not to be.

It no longer fits on a line

If it needs wrapping and re-reading, the four-line loop was clearer.

One transformation, one filter

This is the case comprehensions were made for, and it covers most of them.

The test

Can you say what the comprehension produces in one short sentence, reading left to right, without backtracking? If yes, keep it. If you have to work through it twice, the loop you were avoiding is the clearer code.

Shorter is not the goal. Saying what you want instead of how to get it is the goal, and past a certain length a comprehension stops doing that.

Quick check

You sum a generator expression, then sum the same one again. What comes back the second time?

0 of 9 completed

Real Python runs right here in your browser — nothing to install, nothing sent to a server. The interpreter downloads once the first time you press Run, then stays cached.

To do

Read it left to right: the expression that produces each item, then the loop that supplies them.

squares = [n * n for n in range(1, 6)]
# [1, 4, 9, 16, 25]

The for half is the loop you would have written anyway. The left half is what used to go inside append().

Your task: build squares as the squares of 1 to 5, and doubled as each score doubled. Print both:

[1, 4, 9, 16, 25]
[176, 108, 184, 142]

your_code.py

scores = [88, 54, 92, 71]

squares =
doubled =

print(squares)
print(doubled)

Python

Hint

range(1, 6) gives 1 to 5 — the end is exclusive. For the second, the expression is score * 2 and the source is scores.

Output

To do

This is the shape a comprehension exists for: start empty, walk, append one transformed item, use the result.

Your task: replace the whole loop with a single comprehension. The output must not change:

['Loops', 'Lists', 'Functions']

your_code.py

raw = "loops,  lists ,functions"

cleaned = []
for tag in raw.split(","):
    cleaned.append(tag.strip().title())

print(cleaned)

Python

Hint

The expression is what was inside append(); the for half is the same line you already have. cleaned = [tag.strip().title() for tag in raw.split(",")].

Output

To do

A trailing if filters. Items that fail it never reach the result, so the list comes out shorter.

[n for n in numbers if n % 2 == 0]

Your task: from the rows, build in_stock — the names of the items whose quantity is above zero — and print it with the count:

['tea', 'whisk', 'cloth']
3

The tray has a quantity of 0 and must not appear.

your_code.py

rows = "tea,3 whisk,12 cloth,5 tray,0".split()

in_stock =

print(in_stock)
print(len(in_stock))

Python

Hint

Each row splits into a name and a quantity. Collect row.split(",")[0], and filter on int(row.split(",")[1]) > 0.

Output

To do

Both halves can do work at once. The filter runs first, so the transformation never sees the items being dropped — which matters when the transformation would fail on them.

[name.strip().title() for name in parts if name.strip()]

Without the filter, the blank entries would come through as empty strings.

Your task: the export has blank fields in it. Build tags as the non-blank ones, stripped and lower-cased:

['loops', 'lists', 'functions']
3

your_code.py

raw = "Loops, ,LISTS,,  , Functions "

tags =

print(tags)
print(len(tags))

Python

Hint

Split on the comma, then filter on part.strip() — an empty string is falsy, so the blanks fail the test on their own. The expression is part.strip().lower().

Output

To do

An if/else before the for is the conditional expression from module 2-01. It picks what to collect, and every item still appears — so the result is the same length as the source.

["pass" if s >= 60 else "fail" for s in scores]

This is the distinction that catches people. A trailing if shortens the list; a leading if/else changes the values.

Your task: build both from the same scores. results labels every score, and passing keeps only the ones at or above 60:

['pass', 'fail', 'pass', 'pass']
4
[88, 92, 71]
3

your_code.py

scores = [88, 54, 92, 71]

results =
passing =

print(results)
print(len(results))
print(passing)
print(len(passing))

Python

Hint

results puts the if/else in front of the for and needs an else. passing puts a plain if at the end and has no else at all.

Output

To do

A colon in the middle makes it a dictionary: the key on the left of the colon, the value on the right.

{name: len(name) for name in names}

This is the usual way to reshape data that arrived in the wrong form — turning a list of rows into something you can look up by key.

Your task: build by_name mapping each item name to its quantity as a number, then print it, one lookup, and the number of entries:

{'tea': 3, 'whisk': 12, 'cloth': 5}
12
3

your_code.py

rows = "tea,3 whisk,12 cloth,5".split()

by_name =

print(by_name)
print(by_name["whisk"])
print(len(by_name))

Python

Hint

Unpack inside the for half: for row in rows, then split it. One way is {row.split(",")[0]: int(row.split(",")[1]) for row in rows}.

Output

To do

Curly braces with no colon build a set — so the same shape cleans every item and drops the duplicates, where the loop version needed a seen check and four more lines.

{tag.strip().lower() for tag in raw.split(",")}

Note the order: the cleaning happens first, so " Tea" and "tea" collapse into one entry. Deduplicating before cleaning would have kept both.

Your task: build distinct from the messy export, then print how many raw entries there were, how many survived, and the sorted result:

4
3
['cloth', 'tea', 'whisk']

your_code.py

raw = " Tea, WHISK ,tea,Cloth"

distinct =

print(len(raw.split(",")))
print(len(distinct))
print(sorted(distinct))

Python

Hint

Curly braces, no colon: {part.strip().lower() for part in raw.split(",")}. A set has no order, so sorted() is what makes the printed line stable.

Output

To do

Round brackets make a generator expression. Nothing is computed until something asks, and no list is ever built:

total = sum(int(x) for x in raw.split(","))

On four numbers that saves nothing. On a ten-million-line file it is the difference between working and running out of memory.

The catch: a generator is consumed once. Walk it twice and the second pass finds nothing left, with no error to tell you.

Your task: total the amounts with a generator expression, then demonstrate the catch — assign a generator to once and sum it twice:

3040
3040
0

your_code.py

raw = "1250,480,990,320"

total =

once =

print(total)
print(sum(once))
print(sum(once))

Python

Hint

total = sum(int(x) for x in raw.split(",")) — inside a single function call the brackets are already there. For once, write the generator on its own with round brackets: (int(x) for x in raw.split(",")).

Output

To do

Comprehensions are not always the answer. This one has two levels of nesting, a filter, and a conditional expression, and working out what it produces takes longer than reading the loop it replaced.

The test is simple: can you say what it produces in one sentence, reading left to right, without backtracking? Here, no.

Your task: rewrite it as an ordinary loop that does the same thing. Same output, more lines, and readable:

['TEA', 'whisk', 'CLOTH']
3

The rule it implements: for every in-stock row, take the name, and shout it when the quantity is under ten.

your_code.py

rows = "tea,3 whisk,12 cloth,5 tray,0".split()

names = [p[0].upper() if int(p[1]) < 10 else p[0] for p in [r.split(",") for r in rows] if int(p[1]) > 0]

print(names)
print(len(names))

Python

Hint

Start names as an empty list. Loop over rows, split each into name and quantity, convert the quantity, skip it when it is zero, and append either name.upper() or name depending on whether the quantity is under ten.

Output

The inventory views

To do

One inventory file, five different questions about it. Each answer is one comprehension — which is exactly the case this module is for.

Each real row is name,quantity,unit price. Blank lines and lines starting with # are not rows.

Build these five, with exactly these names:

rows — the real lines, stripped, in file order
names — every item name, in file order
in_stock — the names whose quantity is above zero
value_by_name — a dictionary mapping every in-stock name to quantity times unit price
labels — one label per in-stock item: LOW when the quantity is under ten, otherwise ok

Then total_value, the sum of every line value, worked out with a generator expression rather than by building another list.

Then print exactly six lines:

Rows: 4
Names: ['tea', 'whisk', 'cloth', 'tray']
In stock: ['tea', 'whisk', 'cloth']
Values: {'tea': 3750, 'whisk': 11880, 'cloth': 2250}
Labels: ['LOW', 'ok', 'LOW']
Total: 17880

your_code.py

raw = """tea,3,1250
whisk,12,990

# discontinued
cloth,5,450
tray,0,2200
"""

rows =
names =
in_stock =
value_by_name =
labels =
total_value =

# Six lines of output.

Python

Hint

rows filters raw.splitlines() on a stripped line that is neither empty nor starting with #, collecting the stripped version. Everything else walks rows and splits each one. For value_by_name, the key is the name and the value is int(quantity) * int(price), with a trailing filter on the quantity. labels needs a leading if/else and the same trailing filter, so it stays in step with in_stock. total_value is sum(...) over a generator with no square brackets.

Output

Sign in to open this module

Comprehensions

Four Lines Become One

The loop variable does not leak

Quick check

Two Places an if Can Go, and They Mean Different Things

[x if x > 0 else 0 for x in nums]

[x for x in nums if x > 0]

Quick check

Dictionaries and Sets, Same Idea

Quick check

Generators, and Knowing When to Stop

Two levels of nesting

Side effects

It no longer fits on a line

One transformation, one filter

The test

Quick check

What to collect, and where from

The same four lines, on one

An if at the end drops things

Filter first, then transform

An if at the front changes things

Building a lookup

Clean and deduplicate in one pass

The list you never build

Taking one back apart

The inventory views

Notification

Two Places an `if` Can Go, and They Mean Different Things

`[x if x > 0 else 0 for x in nums]`

`[x for x in nums if x > 0]`