13 min

Simple linear regression — the line

Fit one line to one channel, and read it the way a marketer does: the intercept is base sales, the slope is what a thousand dollars of TV actually buys.

You have $100,000 to put into TV in a new market. Nobody wants a discussion of statistics — they want a number to put in the plan, and a reason to believe it.

Look before you fit established that TV moves with sales. Now we turn that into a line, and the line into two numbers a budget conversation can actually use.

A line claims base sales, plus so much per $1,000

Every straight line through that cloud says the same thing in the same form:

\text{sales} = b_0 + b_1 \times \text{TV budget}

The model. Two numbers, and each one is a business claim.

Two numbers, and each one is a business claim.

Same two numbers, two vocabularies. The second is the one that gets a budget approved.

Term	Statistics calls it	Marketing calls it	The claim it makes
`b₀`	the intercept	base sales	what this market sells with no TV at all
`b₁`	the slope	incremental sales per $1,000	the extra sales one more thousand dollars buys

b₀

Statistics calls it: the intercept
Marketing calls it: base sales
The claim it makes: what this market sells with no TV at all

b₁

Statistics calls it: the slope
Marketing calls it: incremental sales per $1,000
The claim it makes: the extra sales one more thousand dollars buys

So fitting a line is more than curve-drawing. It splits an outcome into what would have happened anyway and what the spending bought. That split is the argument marketing has with finance every quarter.

The second vocabulary is the one that gets a budget approved.

Score a line by adding up its squared errors

Before finding the best line, be able to score a line. Take a market, look at what the line predicted, subtract what actually happened, and you have that market's error. Do it for all 200 and add them up — squared, for reasons we will get to.

\mathrm{SSE}(b_0, b_1) = \sum_{i=1}^{n} \bigl(y_i - (b_0 + b_1 x_i)\bigr)^2

One number for how wrong a line is. Lower is better; zero is impossible.

One number for how wrong a line is.

Score three candidates — a flat line that ignores TV, an over-steep one, and the answer:

Three claims about TV, and what each costs in total squared error.a1-regression/the-line.ipynb

The flat line — TV does nothing, every market sells the average — scores 5,417. The least-squares line scores 2,103, well under half. TV is doing something, and now that is a measured claim rather than an impression.

Notice the middle line: it scores 6,810 — worse than ignoring TV entirely. Believing in a channel too much is not a safer error than believing in it too little. Both are just wrong, and the error is the only thing that knows which is which.

Why squared, and not just the distance? Two reasons, and neither is decoration. Errors above and below the line would cancel out otherwise, and a badly wrong line could score zero. And squaring makes one big miss cost more than several small ones. That is usually what you want. A forecast that is badly wrong in one market does more damage than one slightly off everywhere.

Lower is better, and zero is impossible.

The error curve is a bowl, so there is exactly one right answer

Now sweep it: try every plausible slope, keep the best intercept for each, and plot the error against the slope.

The error as the slope varies. One bowl, one bottom, one answer.a1-regression/the-line.ipynb

That shape is the reason regression has a single right answer rather than an opinion. The error surface is a bowl, so there is exactly one lowest point, and it is the same one no matter where you start looking.

You could find it by hand. Pick a slope, check the error, step downhill, repeat. That is how it is done for models too big to solve directly. It has a name you will meet in Track B: gradient descent. For a straight line we do not need it, because the bottom of this particular bowl has a formula:

b_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}, \qquad b_0 = \bar{y} - b_1 \bar{x}

Least squares, solved. Every regression tool you will ever use runs this.

Read the top line once more. It asks one thing. When a market spends more than average on TV, does it also sell more than average, and by how much? That is all a slope is. The formula is bookkeeping on that question.

Base sales of 7.03k, and 47.5 units per extra $1,000 of TV

The fitted line, extended back to a zero budget.a1-regression/the-line.ipynb

The line crosses the vertical axis at base sales, which is what the market sells on no media at all.

	Value	In words
Base sales `b₀`	7.03k units	what a market sells with no TV budget at all
Incremental `b₁`	0.0475k units per $1,000	about 47.5 extra units for every $1,000 of TV
R²	0.612	TV alone explains most, but nowhere near all, of why markets differ

Base sales b₀

Value: 7.03k units
In words: what a market sells with no TV budget at all

Incremental b₁

Value: 0.0475k units per $1,000
In words: about 47.5 extra units for every $1,000 of TV

R²

Value: 0.612
In words: TV alone explains most, but nowhere near all, of why markets differ

So here is the answer this lesson opened with. At a $100,000 TV budget, the model predicts 11.79k units — 7.03k of which is base and 4.75k of which TV bought.

That is sales, not profit, and not ROI. 47.5 units per $1,000 becomes a return only once you multiply by margin per unit. A channel can be genuinely incremental and still lose money. Regression gives you the physical claim. The finance is yours.

The line crosses the vertical axis at base sales, which is what the market sells on no media at all.

At a $100,000 budget, 60% of predicted sales is base

Everything above collapses into one picture, and it is the one that changes meetings. At any budget, how much of what you sell did you buy?

Predicted sales at four budgets, split into base and incremental.a1-regression/the-line.ipynb

The lower block never moves. At a $100,000 budget it is still 60% of predicted sales. Put another way, most of what that market sells, it would have sold anyway.

That single sentence is why naive return-on-ad-spend overstates marketing's contribution so badly. Divide all the revenue by all the spend and you have quietly claimed the base as well. The regression is what separates them.

Four things this line cannot do yet

It is one channel. Radio and newspaper are still sitting in the file. More than one thing matters is where adding them changes the answer — including for a channel you would not expect.
R² is 0.612. Respectable, and not the whole story. Is it any good? is about what that number does and does not tell you, and about the plot that tells you more.
The cloud bends and the line does not. Look before you fit spotted the flattening at high budgets. A straight line has to average across it, so it will over-promise at the top end. Curves are Regression II.
Base sales is not free sales. It is brand, distribution, habit and everything marketing did in previous years. Calling it “what we get without marketing” is convenient and wrong.

Every one of those is a lesson later in this path.

Run it yourself

The whole lesson is about thirty lines of Python. Read them, copy them, change the channel from tv_spend to radio_spend and watch both numbers move.

The line — from a guess to least squares

content/notebooks/a1-regression/the-line.ipynb

Download Open in Colab

Scores three candidate lines, sweeps the error curve, fits by least squares, and splits the prediction into base and incremental. Runs unchanged in Colab.

Show the code5 cells

Load the data

import numpy as np
import pandas as pd

df = pd.read_csv(CSV)
x = df['tv_spend']     # TV budget, $ thousands
y = df['sales']        # sales, thousands of units

df[['tv_spend', 'sales']].head()

Score a line by its total squared error

def sse(intercept, slope):
    """Sum of squared errors: how wrong this line is, over every market."""
    predicted = intercept + slope * x
    return float(((y - predicted) ** 2).sum())

GUESSES = [(y.mean(), 0.0), (4.0, 0.09), (7.0, 0.0475)]
for intercept, slope in GUESSES:
    print(f'intercept {intercept:5.2f}  slope {slope:.4f}  ->  SSE {sse(intercept, slope):,.0f}')

The error curve, and its one bottom

slopes = np.linspace(0.0, 0.10, 200)
errors = [sse(y.mean() - s * x.mean(), s) for s in slopes]   # best intercept for each slope
best = slopes[int(np.argmin(errors))]
print(f'lowest error at slope ~ {best:.4f}')

Fit the line

slope, intercept = np.polyfit(x, y, 1)
predicted = intercept + slope * x
r2 = 1 - ((y - predicted) ** 2).sum() / ((y - y.mean()) ** 2).sum()

print(f'base sales   (intercept) = {intercept:.2f} thousand units')
print(f'incremental  (slope)     = {slope:.4f} thousand units per $1,000 of TV')
print(f'R2                       = {r2:.3f}')

Base vs incremental at four budgets

budgets = [0, 50, 100, 200]
for budget in budgets:
    incremental = slope * budget
    print(f'${budget:3d}k TV -> {intercept + incremental:5.2f}k units '
          f'({intercept:.2f} base + {incremental:.2f} incremental)')

Next: is it any good? R² is 0.612. We will find out what that promises, and why RMSE is the number to quote to a colleague. And how one plot of the residuals exposes the bend this line hides.

Try this in Expedify →

Related lessons

12 min read

Base rates — what a piece of evidence is actually worth

A face-recognition system that is 99.9% accurate and almost entirely wrong, and a number that sent an innocent woman to prison. Both are the same arithmetic, and it is the arithmetic that decides what any piece of evidence is worth.

Read

10 min read

Confirmation and survivorship — what you never looked for

Two questions about evidence you did not go looking for. One is a rule you have to discover, and one is a pattern in five famous people — and in both, the thing that would have told you the truth is the thing nobody checks.

Read

14 min read

Loss aversion, sunk cost and regression — what it costs you

Four questions you answer about yourself rather than about a scenario, and your own answers are the finding. Then the pattern that makes praise look useless and criticism look like it works, whatever you actually do.

Read