◀ Course contents Part 3 · Module 3-04

Self Joins & UNION

When the other table is this table, and when there is no join at all

Two techniques that look unrelated and solve the same problem: lining rows up when there is no second table to line them up with. One joins a table to itself sideways; the other stacks results on top of each other. Both have a default that quietly returns the wrong number of rows.

Ready?

1

The Other Table Is This Table

users.referred_by holds a user_id — pointing at another row in users. The referrer's name is one join away, and the table on the far side of that join is the same table you started from.

This shape is everywhere once you notice it: employees and their managers, categories and parent categories, comments and the comment they reply to. A column that points back into its own table is a self-referencing key, and reading it needs a self join.

SELECT u.name AS referee, r.name AS referrer
FROM users u
JOIN users r ON r.user_id = u.referred_by;

There is no new syntax. What there is, non-negotiably, is aliases. Name the table twice under different names and from that point on u and r behave like two unrelated tables that happen to contain the same rows.

Without the aliases there is no query

In every earlier module aliases were a convenience. Here they are what makes the query expressible at all: users.user_id = users.referred_by does not mean "join the two copies" — it asks whether a row referred itself. Read your aliases as roles, not abbreviations: u is the person, r is the person who referred them.

The direction is decided entirely by which side of the ON each alias sits on. Flip it and the same two aliases answer the opposite question:

ON r.user_id = u.referred_by   -- r referred u   ("who referred me")
ON u.referred_by = r.user_id   -- same thing, written the other way
ON x.referred_by = u.user_id   -- u referred x   ("who did I refer")

And everything from 3-02 still applies, because a self join is just a join. Four of these twelve users arrived on their own, so an inner join returns 8 rows and a LEFT JOIN returns 12 — with COALESCE(r.name, 'direct') turning the padding into something a reader can act on.

Quick check

An employees table has a manager_id pointing into itself. You self join it with an inner join and get 240 rows from 241 employees. Why the missing one?

2

Pairs, and the Comparison That Deduplicates Them

The second use of a self join has nothing to do with keys. It is for finding pairs of rows that share an attribute: two users on the same plan, two orders on the same day, two employees in the same office.

The obvious version is wrong in two ways at once, and both are easy to miss because the query runs perfectly.

FROM users a
JOIN users b ON a.plan = b.plan     -- 50 rows
1

Every row matches itself

Ada is on the pro plan, and so is Ada. That is a match, so (Ada, Ada) is in the result.

2

Every real pair matches twice

Once as (Ada, Grace) and again as (Grace, Ada). Same fact, two rows, and any count is now double.

Adding a.user_id <> b.user_id is the instinctive fix and it only solves the first problem — 50 rows becomes 38, still twice the truth. The fix for both at once is to demand an order rather than a difference:

ON a.plan = b.plan
AND a.user_id < b.user_id           -- 19 rows

A row cannot be less than itself, so the self-matches go. And of the two orderings of any genuine pair, exactly one satisfies <, so the mirror goes too. One comparison, both problems, and it is worth memorising as a shape rather than re-deriving.

50, 38, 19

Those are the real counts for same-plan pairs in this database, and the gap between them is the point. Nothing errors at 50 or at 38. A "customers who share an address" report built without that comparison reports twice as many matches as exist, and nobody checks a number that was never suspicious.

Quick check

You self join orders on a.ordered_at = b.ordered_at to find orders placed the same day, using a.order_id <> b.order_id. What is wrong?

3

UNION Stacks; UNION ALL Stacks Honestly

Every join in Part 3 has made results wider — more columns, drawn from more tables. Set operators do the other thing. They stack one result on top of another and make it longer.

SELECT 'order' AS kind, o.ordered_at AS happened_on FROM orders o
UNION ALL
SELECT 'event', e.event_at FROM events e
ORDER BY happened_on;

That is the everyday use: two tables recording different kinds of thing, wanted as one list in time order. The 'order' and 'event' are literal columns — constants written into the SELECT list, stored nowhere, labelling which branch each row came from so the stacked result is still readable.

The rules, all enforced

#

Same number of columns

each UNION query must have the same number of columns. No padding, no guessing.

T

Compatible types, matched by position

UNION types text and integer cannot be matched. Column names are irrelevant — only position pairs them.

One ORDER BY, at the very end

It sorts the combined result. An individual branch cannot carry its own.

Names come from the first branch only, which is why writing aliases in the second branch is wasted effort. And because pairing is positional, two branches can compile perfectly and still be wrong: swap two same-typed columns in the second branch and countries arrive filed under a name heading, in silence.

The difference that actually costs people rows

22 rows become 3

UNION

Removes duplicate rows across both branches. To do it, the database must sort or hash the entire combined result — and it deletes rows that were separate real facts which merely looked identical.

22 rows stay 22

UNION ALL

Concatenates and stops. Faster, and it does not silently discard data. This is the one to reach for by default.

Plain UNION is the right choice when you are building a list of distinct values and a repeat is noise. It is the wrong choice when each row is an occurrence — two orders for the same amount on the same day are two orders, and UNION will hand you one.

Quick check

You stack January and February transaction rows with UNION and the total comes out lower than the two months added separately. Why?

4

The Two Set Operators Nobody Teaches

UNION has two siblings. They follow the same column rules, they deduplicate the same way, and they are genuinely useful — they just turn up in tutorials far less often than they deserve.

INTERSECT

Rows appearing in both results. "Users who have ordered and done something in the app." It can only ever make a result smaller.

EXCEPT

Rows in the first result that are absent from the second. Order matters enormously: A EXCEPT B is not B EXCEPT A.

SELECT user_id FROM events
EXCEPT
SELECT user_id FROM orders;      -- acted, never bought

That is the anti-join from 3-02, in three lines instead of four and without the IS NULL subtlety. When you need only the key — which ids are in this list and not that one — EXCEPT says so more plainly than a LEFT JOIN can.

When you need columns from the other table as well, the LEFT JOIN is still the tool: EXCEPT compares whole rows and hands back only what both branches selected.

Postgres uses the standard spelling for all three. Some databases call EXCEPT MINUS instead, which is worth knowing if you ever move.

Set operator, or join?

A quick way to decide. If the answer needs columns from both sides on one row, that is a join. If the answer is one list, and the question is which rows belong in it, that is a set operator. Stacking is not a weaker join — it is a different question.

Quick check

You want the ids of customers who bought in 2023 and in 2024. Which is the most direct?

0 of 9 completed

Loading the tables…

01

To do

users.referred_by holds a user_id — of another row in users. The referrer's name is one join away, and the table on the other side of that join is the same table.

Nothing about the syntax is special. What is mandatory is aliases: name the table twice, differently, and from then on u and r behave like two unrelated tables that happen to hold the same rows.

FROM users u
JOIN users r ON r.user_id = u.referred_by

Read it as "u is the person, r is the person who referred them". Without the two aliases there is no way to write the ON at all — users.user_id = users.referred_by asks whether a row referred itself.

Your task: return each referred user's name as referee and their referrer's name as referrer, ordered by the referee's user_id.

query.sql
PostgreSQL
Hint

r.name AS referrer, and the condition is r.user_id = u.referred_by — the referrer's id matches the value stored on the referee's row.

Output

      
    02

    To do

    The inner join in the last station answered "who referred whom". A different question — "how did each user arrive?" — is about every user, and four of them arrived on their own.

    Everything from 3-02 applies unchanged. A self join is still a join, so LEFT JOIN keeps the unmatched rows and pads the right-hand alias with NULL.

    COALESCE then turns that NULL into something a reader can act on. A blank cell in a "referred by" column is ambiguous — missing data, or nobody? — and the word direct is neither.

    Your task: return every user's name and their referrer's name as referrer, using the text direct where there was no referrer. Order by u.user_id.

    query.sql
    PostgreSQL
    Hint

    COALESCE(r.name, 'direct') AS referrer. The single quotes make it a text literal, not a column name.

    Output
    
          
      03

      To do

      The same two aliases, the same one column, read in the opposite direction. Instead of "who referred me", ask "who did I refer" — and the ON flips.

      ON u.referred_by = r.user_id   -- r is the referrer, u is their referee

      Now r is the row you keep and group by, and each u is somebody they brought in. A LEFT JOIN keeps the users who referred nobody, and — exactly as in 3-02 — COUNT has to count a column, not rows, or everyone who referred nobody reports 1.

      Your task: return every user's name and how many people they referred as referred. Order by referred descending, then name.

      query.sql
      PostgreSQL
      Hint

      ON u.referred_by = r.user_id, and COUNT(u.user_id) — counting the referee's id, so a user who referred nobody scores 0 rather than 1.

      Output
      
            
        04

        To do

        The other use of a self join: finding pairs of rows that share something. Two users on the same plan, two orders on the same day, two employees in the same office.

        The obvious version is wrong twice over. ON a.plan = b.plan matches every user with themselves, and matches each real pair twice — once as (Ada, Grace) and again as (Grace, Ada).

        Adding a.user_id <> b.user_id fixes only the first problem. The fix for both at once is to demand an order:

        ON a.plan = b.plan AND a.user_id < b.user_id

        A row cannot be less than itself, so self-matches go; and of the two orderings of any real pair, exactly one satisfies <. Without it there are 50 rows here. With it, 19.

        Your task: return every pair of users on the same plan as person_a, person_b and plan, each pair once, ordered by a.user_id then b.user_id.

        query.sql
        PostgreSQL
        Hint

        Add AND a.user_id < b.user_id to the ON. Using <> instead leaves every pair in twice.

        Output
        
              
          05

          To do

          Every join so far made results wider — more columns, from more tables. UNION does the other thing: it stacks one result on top of another, making it longer.

          SELECT plan FROM users
          UNION
          SELECT plan FROM subscriptions;

          Three rules, all enforced. Both sides need the same number of columns, in the same order, with compatible types — Postgres will say each UNION query must have the same number of columns or UNION types text and integer cannot be matched rather than guessing. Column names come from the first branch.

          And the part that surprises people: UNION removes duplicates, across both branches. 22 plan values go in; 3 distinct ones come out.

          Your task: return every distinct plan name appearing in either users or subscriptions, as plan, ordered by plan.

          query.sql
          PostgreSQL
          Hint

          Put UNION between the two SELECTs. One ORDER BY at the very end orders the combined result — it cannot go on an individual branch.

          Output
          
                
            06

            To do

            UNION deduplicates. That sounds harmless and is not: to remove duplicates the database must sort or hash the entire combined result, and it removes rows that were genuinely distinct records which merely happened to look identical.

            UNION ALL concatenates and stops. It is faster, and it does not silently delete data.

            ALL

            Reach for this by default

            Stacking rows that represent separate real things — two months of orders, two regions' sales. Two identical rows are two facts.

            UNION

            Only when you mean it

            Building a list of distinct values, where a repeat is noise rather than a second occurrence.

            Your task: return every plan value from both tables, keeping duplicates, as plan, ordered by plan.

            query.sql
            PostgreSQL
            Hint

            One word: UNION ALL. 12 rows from users plus 10 from subscriptions is 22.

            Output
            
                  
              07

              To do

              The everyday use of UNION ALL: two tables recording different kinds of thing, wanted as one list in time order. Orders and events have nothing in common structurally — but "what did this user do, in order" needs both.

              The trick is a literal column that labels which branch each row came from. It is not stored anywhere; you write it into the SELECT list, and it survives the stacking.

              SELECT 'order' AS kind, o.ordered_at AS happened_on FROM orders o
              UNION ALL
              SELECT 'event', e.event_at FROM events e

              Only the first branch's aliases matter — the second branch's column names are ignored entirely, so there is no point writing them. The ORDER BY goes once, at the end, and applies to the whole stack.

              Your task: build that timeline for user 1: kind (order or event) and happened_on, ordered by happened_on then kind.

              query.sql
              PostgreSQL
              Hint

              'order' in single quotes is a text literal. The second branch is SELECT 'event', e.event_at FROM events e WHERE e.user_id = 1.

              Output
              
                    
                08

                To do

                Two more set operators, both properly useful and both rarely taught. They follow the same column rules as UNION, and they deduplicate the same way.

                INTERSECT

                Rows appearing in both results. "Users who have ordered and done something in the app" — 10 of them.

                EXCEPT

                Rows in the first result and not the second. Order matters: A EXCEPT B is not B EXCEPT A.

                EXCEPT is the readable alternative to the anti-join from 3-02 when you only need the key and not the other table's columns. Some databases spell it MINUS; Postgres uses the standard word.

                Your task: return the user_id of every user who has an event but has never placed an order, ordered by user_id.

                query.sql
                PostgreSQL
                Hint

                EXCEPT between them. Events first, because you want the ones in events that are missing from orders.

                Output
                
                      
                  09

                  To do

                  The constraints are worth meeting deliberately once, so the errors are recognisable later.

                  #

                  Same column count

                  each UNION query must have the same number of columns. No padding, no guessing.

                  T

                  Compatible types, by position

                  UNION types text and integer cannot be matched. Position decides what pairs with what — names are irrelevant.

                  One ORDER BY, at the end

                  It sorts the combined result. A branch cannot have its own.

                  Because pairing is positional, two branches can compile perfectly and still be wrong: swap two columns of matching type in the second branch and you get countries under a name heading, silently.

                  Your task: one list of everyone the business deals with. Return name and country from users, stacked with the product name and the literal 'catalogue' as country from products. Keep duplicates, order by name.

                  query.sql
                  PostgreSQL
                  Hint

                  SELECT name, 'catalogue' FROM products — two columns, both text, in the same order as the first branch.

                  Output
                  
                        

                    The referral report

                    To do

                    Growth wants to know how the user base actually arrived: who was brought in by whom, and who is bringing people in. Both facts live in the same column of the same table, read in opposite directions — so this needs the table joined to itself twice.

                    Your task: one row for every user, with:

                    • name — the user's name
                    • referrer — the name of whoever referred them, or the text direct if nobody did
                    • referred — how many users they referred, 0 if none

                    Order by referred descending, then name.

                    Three aliases of one table, then. The user, the person above them, and the people below them. Both of those joins have to be outer ones or you will lose the four users nobody referred, the six who referred nobody, or both.

                    query.sql
                    PostgreSQL
                    Hint

                    r joins on r.user_id = u.referred_by; x joins on x.referred_by = u.user_id. Then COALESCE(r.name, 'direct') AS referrer and COUNT(x.user_id) AS referred, grouped by u.user_id, u.name, r.name.

                    Output
                    
                          

                      Notification