14 min

Multiple regression — more than one thing matters

Radio and newspaper join the model. R² jumps from 0.61 to 0.90. And one of the three channels turns out to have been contributing nothing at all, which is the most useful thing this path has to say.

Every model so far has answered one question: what does TV buy? But nobody's budget has one line in it. The real question is the allocation — of the next $1,000, which channel should get it?. And a one-channel model cannot answer that, because it has never been asked to weigh one channel against another.

So put all three in at once.

\text{sales} = b_0 + b_1 \,\text{TV} + b_2 \,\text{radio} + b_3 \,\text{newspaper}

One term per channel. The fitting method does not change at all.

Mechanically this is no harder than the straight line. The same least squares, minimising the same squared error, over three slopes instead of one. What changes is not the arithmetic. It is what a coefficient means, and that change is the whole lesson.

Newspaper's coefficient goes from 0.0547 to −0.001 when radio joins the model

Each channel on its own first, then all three at once, and put the two answers side by side.

What each $1,000 buys — fitted alone, and fitted alongside the others.a1-regression/more-than-one-thing-matters.ipynb

Not smaller. Gone.

Extra sales per $1,000, in thousands of units.

Channel	Fitted alone	With the others	What happened
TV	0.0475	0.0458	essentially unchanged
Radio	0.2025	0.1885	still strong, somewhat reduced
Newspaper	0.0547	-0.001	gone — zero, and very slightly negative

Fitted alone: 0.0475
With the others: 0.0458
What happened: essentially unchanged

Radio

Fitted alone: 0.2025
With the others: 0.1885
What happened: still strong, somewhat reduced

Newspaper

Fitted alone: 0.0547
With the others: -0.001
What happened: gone — zero, and very slightly negative

Newspaper went from a positive, believable 0.0547 to -0.001. Not smaller. Gone. And this is the channel taking 15% of the media budget — $30.6k per market, which is more than the $23.3k going to radio.

Not smaller. Gone.

The two numbers answer two different questions

The two numbers answer two different questions, and once you see that, nothing here is mysterious.

	The question it answers
Fitted alone	Do markets that spend more on newspaper sell more? — Yes, somewhat.
Fitted with the others	Among markets spending the same on TV and radio, do the ones spending more on newspaper sell more?. No.

Fitted alone

The question it answers: Do markets that spend more on newspaper sell more? — Yes, somewhat.

Fitted with the others

The question it answers: Among markets spending the same on TV and radio, do the ones spending more on newspaper sell more?. No.

That second sentence is what holding the others constant means, and it is the single most valuable thing a multiple regression does. The first question is about correlation in the world. The second is about what newspaper contributes once you already know the rest of the plan. That is the only version a budget meeting cares about.

And the reason the two answers differ is visible in one chart:

The two budgets, plotted against each other.a1-regression/more-than-one-thing-matters.ipynb

Newspaper and radio budgets travel together (0.35). Markets that buy one tend to buy the other, presumably because both are planned by the same person under the same regional brief. So when newspaper was fitted alone, it was quietly taking credit for radio's work. Against TV, newspaper is nearly independent (0.06), which is why TV's coefficient barely moved.

This is the trap L2 refused to spring. Newspaper's 0.23 correlation with sales was real, positive, and completely misleading. A single-variable correlation is not evidence about a channel. It is evidence about a channel plus everything that channel travels with. That is why the shortlist in Look before you fit was called a shortlist.

Radio adds real R². Newspaper adds less than a random column would.

Build the model up one channel at a time and watch R².

R² as each channel joins the model.a1-regression/more-than-one-thing-matters.ipynb

R² cannot fall when a column is added, and newspaper still managed almost nothing.

TV alone: 0.612. The model from The line.
Add radio: 0.897 — a gain of 0.285. Enormous. Most of what the residual plot in Is it any good? could not explain was radio all along.
Add newspaper: 0.897 — a gain of 0. Nothing, to three decimal places.

Remember from Is it any good? that R² cannot go down when you add a column. Even a column of random numbers nudges it up. Newspaper could not manage even that. It is an absent channel in this data rather than a weak one.

R² cannot go down when a column is added, and newspaper still managed almost nothing.

Base sales drops from 7.03k to 2.94k once all three channels are in

Coefficients become useful the moment you multiply them by what was actually spent. That turns a slope into a contribution: how many units this channel put on the board in the average market.

\hat{y} = \underbrace{b_0}_{\text{base}} + \underbrace{b_1 x_1}_{\text{TV}} + \underbrace{b_2 x_2}_{\text{radio}} + \underbrace{b_3 x_3}_{\text{newspaper}}

The decomposition. Every term is a claim about where the sales came from.

Where the average market's predicted sales come from.a1-regression/more-than-one-thing-matters.ipynb

Base sales is whatever the model cannot attribute, so it shrinks every time you explain more.

The average market, decomposed.

	Contribution	Budget
Base	2.94k units	—
TV	6.73k units	$147.0k
Radio	4.39k units	$23.3k
Newspaper	-0.03k units	$30.6k
Total	14.02k units	—

Base

Contribution: 2.94k units
Budget: —

Contribution: 6.73k units
Budget: $147.0k

Radio

Contribution: 4.39k units
Budget: $23.3k

Newspaper

Contribution: -0.03k units
Budget: $30.6k

Total

Contribution: 14.02k units
Budget: —

And notice what happened to base sales. The one-channel model in The line put it at 7.03k units. With all three channels it is 2.94k — less than half. The TV-only model was crediting "what we'd have sold anyway" with work radio was doing. Base sales is a property of the model, not of the business, and quoting it from a one-channel model overstates it badly. Whenever someone tells you their base is X, the right question is: base, after accounting for what?

Base sales is whatever the model cannot attribute, so it shrinks every time you explain more.

$30.6k per market, 15% of spend, contributing −0.03k units

This is what the whole path has been for. $30.6k per market, 15% of media spend, contributing -0.03k units. On this evidence the recommendation writes itself. And it is worth being careful about exactly how far the evidence goes.

What the model does support: newspaper is not earning its place in this plan. Reallocating it is the obvious first test, and radio is the obvious destination at 0.1885 against TV's 0.0458.
What it does not support: a specific promised return. Moving newspaper's budget into radio would take the average market to $53.8k of radio. And the largest radio budget the model has ever seen is $49.6k. Multiplying a coefficient out past the data is extrapolation. The bend spotted in Look before you fit is a warning about precisely that: channels saturate. Expect less than the arithmetic promises.
And it is still not causal. Nobody randomised these budgets. The honest recommendation is test it — hold newspaper flat in some markets, cut it in others, and read the difference. That experiment is cheap now that you know which line to point it at.

The pattern to carry out of this lesson. A variable's value is not a property of the variable. It depends entirely on what else is in the model. So "is this channel working?" is a question you cannot answer one channel at a time, however clean the correlation looks.

Run it yourself

The fit() helper takes any list of columns, so you can ask this of any combination. Try ['radio_spend', 'newspaper_spend'] with TV left out and watch newspaper come back to life.

More than one thing matters — three channels at once

content/notebooks/a1-regression/more-than-one-thing-matters.ipynb

Download Open in Colab

Fits each channel alone and all three together, walks R² up the ladder, shows why newspaper collapses, and decomposes the average market into base and per-channel contributions. Runs unchanged in Colab.

Show the code5 cells

One channel at a time, then all three

import numpy as np
import pandas as pd

df = pd.read_csv(CSV)
y = df['sales']

def fit(columns):
    """Least squares on any set of columns. Returns coefficients and R²."""
    X = np.column_stack([np.ones(len(df))] + [df[c] for c in columns])
    coef, *_ = np.linalg.lstsq(X, y, rcond=None)
    residual = y - X @ coef
    r2 = 1 - (residual ** 2).sum() / ((y - y.mean()) ** 2).sum()
    return dict(zip(['intercept'] + columns, coef)), float(r2)

for channel in ['tv_spend', 'radio_spend', 'newspaper_spend']:
    coef, r2 = fit([channel])
    print(f'{channel:16s} alone: coefficient {coef[channel]:+.4f}   R2 {r2:.3f}')

All three together — and what happens to newspaper

CHANNELS = ['tv_spend', 'radio_spend', 'newspaper_spend']
full, r2_full = fit(CHANNELS)

for channel in CHANNELS:
    alone = fit([channel])[0][channel]
    print(f'{channel:16s} alone {alone:+.4f}   with the others {full[channel]:+.4f}')
print(f'\nR2 with all three: {r2_full:.3f}')

R² as each channel joins

steps = [['tv_spend'],
         ['tv_spend', 'radio_spend'],
         ['tv_spend', 'radio_spend', 'newspaper_spend']]

previous = 0.0
for columns in steps:
    _, r2 = fit(columns)
    print(f'{len(columns)} channel(s): R2 {r2:.3f}   (+{r2 - previous:.3f})')
    previous = r2

Newspaper and radio budgets move together

print(f"newspaper vs radio: {df['newspaper_spend'].corr(df['radio_spend']):+.2f}")
print(f"newspaper vs TV:    {df['newspaper_spend'].corr(df['tv_spend']):+.2f}")

Contribution in the average market

means = df[CHANNELS].mean()
contribution = {c: full[c] * means[c] for c in CHANNELS}
total = full['intercept'] + sum(contribution.values())

print(f"base                 {full['intercept']:6.2f}k units")
for channel, amount in contribution.items():
    print(f'{channel:20s} {amount:6.2f}k units   (${means[channel]:.0f}k spent)')
print(f'predicted total      {total:6.2f}k units')

Next: we found newspaper by looking. Next is how you find it on purpose. Deciding which features earn their place, and why R² cannot be the judge. And what to use instead when two models both look fine.

Try this in Expedify →

Related lessons

12 min read

Base rates — what a piece of evidence is actually worth

A face-recognition system that is 99.9% accurate and almost entirely wrong, and a number that sent an innocent woman to prison. Both are the same arithmetic, and it is the arithmetic that decides what any piece of evidence is worth.

Read

10 min read

Confirmation and survivorship — what you never looked for

Two questions about evidence you did not go looking for. One is a rule you have to discover, and one is a pattern in five famous people — and in both, the thing that would have told you the truth is the thing nobody checks.

Read

14 min read

Loss aversion, sunk cost and regression — what it costs you

Four questions you answer about yourself rather than about a scenario, and your own answers are the finding. Then the pattern that makes praise look useless and criticism look like it works, whatever you actually do.

Read