Unsupervised learning — when nobody tells you anything
Take the target away. No coach, no score, no right answer to check against. The robot shoots into an empty field — and something can still be found out from where the arrows land.
Being told the answer ended on the assumption underneath supervised learning. Somebody could see the target and was willing to say so.
Most of the data in the world has nobody standing behind it. So take the target away entirely. No coach, no score, no right answer to check against.
Three kinds of arrow, and nobody marked which is which
The robot is shooting three kinds of arrow and does not know it.
- Heavy ones. They fall low.
- Light ones. They fly high.
- Warped ones. They veer left.
- And the wobble, still. 4 centimetres of scatter on every shot, exactly as before.
The causes are real whether or not anybody records them. That sentence is the reason this kind of learning works at all. Customers arrange themselves into groups long before anyone gets round to naming the segments.
Ninety arrows, no labels, no target, no score
That is the entire input. 90 pairs of numbers, 30 of each kind, and no column saying which is which.
You can already see the groups, and nobody told you there were three. You got that from the picture. A method can get it from the numbers the same way.
Shoot into the empty field and press Find the groups
The same range with the target taken off it. Shoot fifty or so, then press the button. It runs over the landing points and nothing else.
- Arrows
- 0
- Groups found
- —
Nothing in that button knows what a heavy arrow is. It knows where ninety dots are, and that some are nearer to each other than to the rest. Everything unsupervised learning does is that idea, applied to numbers with more columns.
It files 98% of the arrows with their own kind, having been told nothing
98 per cent of the arrows ended up filed with their own kind. Across a dozen fresh rounds it averages 95 per cent. Guessing would get 33 per cent.
That is a real finding about a real cause, obtained with no labels and no answer key. It works on the data you already have, rather than data somebody would have had to sit down and mark.
How many groups there are is a question the data cannot answer
Nothing above told it there were three groups. It was asked for three. So ask for two, four, six and eight, and score each by how tight the groups are.
Asking for the third group bought 4,866. Asking for the fourth bought 315. The score improves forever, so it can never tell you when to stop.
The bend at three is a real hint, and you are reading it off a chart with your eye. The method did not hand it to you. How many segments a business has is argued for by people.
When the causes overlap it returns confident nonsense
Everything so far used three causes that sit far apart. Push them together, with the same method and the same three groups asked for.
At a quarter of the separation it files 55 per cent correctly, against 33 for pure guessing. And it says nothing. There is no error, no warning and no low-confidence flag.
Clustering does not fail loudly. It returns confident nonsense at the same volume as a good answer. The groups still come back, still have centres, and still look like a slide. Somebody has to check whether they mean anything.
Unsupervised learning finds structure where nobody supplied an answer
No labels to learn from, and no accuracy to report, because there is nothing to be right about.
The purity figures above exist only because this is a simulation and we kept the truth aside to score ourselves. On real data nobody has that. So the groups have to be judged by whether they are useful.
It found three groups. It did not find that one of them is the heavy arrows.
Nothing in it knows what heavy means, and nothing in it could. The names are ours. The groups are the data's.
Which sets up the last of the three. Told the exact miss, the robot learns quickly. Told nothing, it can find out how its arrows differ. Told only whether it hit, with no direction, it can still get there. That is the next lesson.
content/notebooks/what-learning-is/when-nobody-tells-you-anything.ipynb
Move the three causes closer together, or ask for a different number of groups. Worth trying: ask for eight groups on three causes and look at what it hands back.
Show the code6 cells
import random
WOBBLE = 4.0 # cm: the same irreducible scatter as the other lessons
ARROWS_EACH = 30
# (name, direction, tension) — the cause, in centimetres.
KINDS = [
('heavy', 0.0, -12.0), # falls low
('light', 2.0, 10.0), # flies high
('warped', -13.0, -1.0), # veers left
]
def shoot_all(scale=1.0, seed=3):
"""Every arrow: where it landed, and (kept aside, never used) which kind it was."""
rng = random.Random(seed)
points, truth = [], []
for kind, (name, dx, dy) in enumerate(KINDS):
for _ in range(ARROWS_EACH):
points.append((dx * scale + rng.gauss(0, WOBBLE),
dy * scale + rng.gauss(0, WOBBLE)))
truth.append(kind)
return points, truth
points, truth = shoot_all()
record('kinds', len(KINDS))
record('arrows_each', ARROWS_EACH)
record('arrows_total', len(points))
record('wobble_cm', WOBBLE)import matplotlib.pyplot as plt
import matplotlib.patches as patches
def draw_field(ax, shots, colours=None, archer=True):
"""The same range as lesson one, with the target taken off it."""
if archer:
ax.plot([0, 0], [-54, -42], color='#3a3a3a', lw=1.5)
ax.plot([-4, 0, 4], [-60, -54, -60], color='#3a3a3a', lw=1.5)
ax.plot([-8, 0, 5], [-45, -44, -42], color='#3a3a3a', lw=1.5)
ax.add_artist(patches.Circle((0, -38), 3, fc='none', ec='#3a3a3a', lw=1.5))
ax.add_artist(patches.Arc((-9, -45), 7, 15, theta1=90, theta2=270, color='#3a3a3a', lw=1.5))
ax.scatter([p[0] for p in shots], [p[1] for p in shots], s=40,
color=colours if colours is not None else '#8a8a94', alpha=0.8, zorder=2)
ax.set_xlim(-34, 30); ax.set_ylim(-64, 30); ax.set_aspect('equal')
ax.set_xticks([]); ax.set_yticks([]); ax.grid(False)
for side in ('top', 'right', 'bottom', 'left'):
ax.spines[side].set_visible(False)
save_fig('where-they-fell', lambda ax: draw_field(ax, points), figsize=(5.0, 5.6))def k_means(pts, k, rounds=30):
ordered = sorted(pts)
centres = [ordered[min(len(ordered) - 1, int((i + 0.5) / k * len(ordered)))] for i in range(k)]
assign = [0] * len(pts)
for _ in range(rounds):
for i, p in enumerate(pts):
assign[i] = min(range(k),
key=lambda c: (p[0] - centres[c][0]) ** 2 + (p[1] - centres[c][1]) ** 2)
for c in range(k):
mine = [p for p, a in zip(pts, assign) if a == c]
if mine:
centres[c] = (sum(p[0] for p in mine) / len(mine),
sum(p[1] for p in mine) / len(mine))
return assign, centres
def purity(assign, truth, k):
"""Share of arrows filed with the kind that dominates their group."""
total = 0
for c in range(k):
mine = [t for a, t in zip(assign, truth) if a == c]
if mine:
total += max(mine.count(t) for t in set(mine))
return total / len(truth)
found, centres = k_means(points, len(KINDS))
score = purity(found, truth, len(KINDS))
record('purity_pct', round(100 * score))
print(f'{100 * score:.0f}% of arrows were filed with their own kind — and nothing was told')def plot_found(ax):
tones = ['#ee785b', '#2f9e6e', '#3b6fd4']
draw_field(ax, points, colours=[tones[a] for a in found])
for c in centres:
ax.scatter([c[0]], [c[1]], marker='x', s=90, color='#3a3a3a', linewidths=2, zorder=3)
ax.legend(handles=[plt.Line2D([], [], marker='o', ls='', color=tones[i], label=f'group {i + 1}')
for i in range(len(KINDS))], frameon=False, fontsize=10, loc='upper right')
save_fig('the-groups-it-found', plot_found, figsize=(5.0, 5.6))def tightness(pts, assign, centres):
return sum((p[0] - centres[a][0]) ** 2 + (p[1] - centres[a][1]) ** 2
for p, a in zip(pts, assign))
ks = [1, 2, 3, 4, 5, 6, 8]
scores = []
for k in ks:
a, c = k_means(points, k)
scores.append(tightness(points, a, c))
for k, s in zip(ks, scores):
print(f'{k} groups -> tightness {s:,.0f}')
record('tight_two', round(scores[ks.index(2)]))
record('tight_three', round(scores[ks.index(3)]))
record('tight_eight', round(scores[ks.index(8)]))
record('k_asked_max', ks[-1])
# The bend: what asking for the third group bought, against the fourth.
record('drop_to_three', f'{round(scores[ks.index(2)] - scores[ks.index(3)]):,}')
record('drop_to_four', f'{round(scores[ks.index(3)] - scores[ks.index(4)]):,}')
def plot_ks(ax):
ax.plot(ks, scores, color='#ee785b', linewidth=2.4, marker='o', markersize=6)
ax.set_xlabel('groups asked for')
ax.set_ylabel('total distance to own centre')
ax.set_ylim(0, max(scores) * 1.1)
ax.spines[['top', 'right']].set_visible(False)
save_fig('how-many-groups', plot_ks, figsize=(7.0, 3.8))scales = [1.0, 0.6, 0.4, 0.25]
purities = []
for sc in scales:
runs = []
for seed in range(12):
pts, tr = shoot_all(scale=sc, seed=seed)
a, _ = k_means(pts, len(KINDS))
runs.append(purity(a, tr, len(KINDS)))
purities.append(sum(runs) / len(runs))
for sc, pu in zip(scales, purities):
print(f'causes at {sc:.2f} x apart -> {100 * pu:.0f}% filed correctly')
record('purity_far', round(100 * purities[0]))
record('purity_close', round(100 * purities[-1]))
record('close_scale', scales[-1])
record('guessing_pct', round(100 / len(KINDS)))
def plot_overlap(ax):
labels = [f'{s:.2f}x' for s in scales]
bars = ax.bar(labels, [100 * p for p in purities],
color=['#ee785b'] + ['#9aa1ab'] * (len(scales) - 1))
for b, p in zip(bars, purities):
ax.text(b.get_x() + b.get_width() / 2, 100 * p + 1.5, f'{100 * p:.0f}%',
ha='center', fontsize=10)
ax.axhline(100 / len(KINDS), color='#6b7280', linestyle=':', linewidth=1.2)
ax.text(-0.42, 100 / len(KINDS) + 2.5, 'what guessing would get',
fontsize=9.5, color='#6b7280')
ax.set_ylabel('filed with their own kind (%)')
ax.set_xlabel('how far apart the three causes are')
ax.set_ylim(0, 108)
ax.spines[['top', 'right']].set_visible(False)
save_fig('when-they-overlap', plot_overlap, figsize=(7.0, 3.8))Related lessons
Base rates — what a piece of evidence is actually worth
A face-recognition system that is 99.9% accurate and almost entirely wrong, and a number that sent an innocent woman to prison. Both are the same arithmetic, and it is the arithmetic that decides what any piece of evidence is worth.
ReadConfirmation and survivorship — what you never looked for
Two questions about evidence you did not go looking for. One is a rule you have to discover, and one is a pattern in five famous people — and in both, the thing that would have told you the truth is the thing nobody checks.
ReadLoss aversion, sunk cost and regression — what it costs you
Four questions you answer about yourself rather than about a scenario, and your own answers are the finding. Then the pattern that makes praise look useless and criticism look like it works, whatever you actually do.
Read