{ "cells": [ { "cell_type": "markdown", "id": "5974fc6b", "metadata": {}, "source": [ "# What learning is · The robot with a bow\n", "\n", "A robot that shoots arrows. It has a bad habit — it pulls left and high, the\n", "way a person does — and it cannot see the target.\n", "\n", "Two robots, identical in every way except one: after each arrow, one of them\n", "is **told where the arrow landed** and the other is told nothing. Everything\n", "in this notebook is that one difference." ] }, { "cell_type": "code", "execution_count": 1, "id": "dbcb16c5", "metadata": { "execution": { "iopub.execute_input": "2026-08-19T08:39:43.640877Z", "iopub.status.busy": "2026-08-19T08:39:43.640741Z", "iopub.status.idle": "2026-08-19T08:39:44.023580Z", "shell.execute_reply": "2026-08-19T08:39:44.023166Z" } }, "outputs": [], "source": [ "# hide — plumbing so this file runs both in the repo and in Colab.\n", "try:\n", " from _figkit import save_fig, record\n", "except ImportError: # Colab — no repo, no problem\n", " def save_fig(name, plot, **kw):\n", " import matplotlib.pyplot as plt\n", " fig, ax = plt.subplots(figsize=kw.get('figsize', (7, 4.2))); plot(ax); plt.show()\n", " def record(key, value):\n", " print(f'{key} = {value}'); return value" ] }, { "cell_type": "markdown", "id": "53f31424", "metadata": {}, "source": [ "## 1. The bow\n", "\n", "An arrow lands where it is aimed, plus two things:\n", "\n", "* a **bias** — the robot's habit, the same push on every shot;\n", "* a **wobble** — random, different every time, and nothing can remove it.\n", "\n", "The distinction matters more than it looks. Learning can fix the first and can\n", "never fix the second, and most disappointment with a model is somebody\n", "expecting it to fix the second." ] }, { "cell_type": "code", "execution_count": 2, "id": "5f82e680", "metadata": { "execution": { "iopub.execute_input": "2026-08-19T08:39:44.025522Z", "iopub.status.busy": "2026-08-19T08:39:44.025372Z", "iopub.status.idle": "2026-08-19T08:39:44.031807Z", "shell.execute_reply": "2026-08-19T08:39:44.030883Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "bias_cm = 13.0\n", "bias_direction = -11.0\n", "bias_tension = 7.0\n", "wobble_cm = 4.0\n", "arrows = 60\n", "trials = 400\n", "gold_cm = 10\n" ] }, { "data": { "text/plain": [ "10" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#| caption: One bow, one habit, one unavoidable wobble\n", "import random\n", "\n", "# The robot has exactly two settings it can change, and they are its\n", "# PARAMETERS: how it is pointed, and how hard it is drawn.\n", "# x — direction: negative is left of the bullseye, positive is right\n", "# y — tension: negative falls short and low, positive flies high\n", "# Its habit is one fixed error on each: pointed left, drawn too hard.\n", "BIAS = (-11.0, 7.0) # cm: (direction, tension), every single shot\n", "WOBBLE = 4.0 # cm: irreducible\n", "ARROWS = 60\n", "TRIALS = 400\n", "GOLD = 10.0 # cm: inside this ring counts as a hit\n", "\n", "def shoot(aim, rng):\n", " \"\"\"Where the arrow actually lands, given where it was aimed.\"\"\"\n", " return (aim[0] + BIAS[0] + rng.gauss(0, WOBBLE),\n", " aim[1] + BIAS[1] + rng.gauss(0, WOBBLE))\n", "\n", "def miss_by(shot):\n", " return (shot[0] ** 2 + shot[1] ** 2) ** 0.5\n", "\n", "record('bias_cm', round((BIAS[0] ** 2 + BIAS[1] ** 2) ** 0.5, 1))\n", "record('bias_direction', BIAS[0])\n", "record('bias_tension', BIAS[1])\n", "record('wobble_cm', WOBBLE)\n", "record('arrows', ARROWS)\n", "record('trials', TRIALS)\n", "record('gold_cm', int(GOLD))" ] }, { "cell_type": "markdown", "id": "efede32b", "metadata": {}, "source": [ "### What that looks like\n", "\n", "Before a single number: ten arrows, aimed straight at the bullseye, from a bow\n", "nobody has corrected. The archer is drawn from behind, and the whole of the rest\n", "of this notebook is a way of measuring what you are about to see." ] }, { "cell_type": "code", "execution_count": 3, "id": "3d91412e", "metadata": { "execution": { "iopub.execute_input": "2026-08-19T08:39:44.034648Z", "iopub.status.busy": "2026-08-19T08:39:44.034513Z", "iopub.status.idle": "2026-08-19T08:39:44.093788Z", "shell.execute_reply": "2026-08-19T08:39:44.093593Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "figure the-range -> the-robot-with-a-bow.the-range.{light,dark}.svg\n" ] } ], "source": [ "#| caption: The range, drawn the same way the robot sees it\n", "import matplotlib.pyplot as plt\n", "import matplotlib.patches as patches\n", "\n", "def draw_range(ax, shots=(), gold=GOLD, archer=True, colour='#ee785b', hollow=False):\n", " \"\"\"Target, archer and arrows — the picture every number in this notebook is about.\"\"\"\n", " for r, shade in zip((30, 20, gold), ('#f1f2f4', '#e4e6ea', '#d5d8dd')):\n", " ax.add_artist(patches.Circle((0, 0), r, facecolor=shade, edgecolor='none', zorder=0))\n", " ax.plot([0], [0], '+', color='#3a3a3a', markersize=9, mew=1.4, zorder=1)\n", " if archer: # standing below, seen from behind\n", " ax.plot([0, 0], [-54, -42], color='#3a3a3a', lw=1.5) # body\n", " ax.plot([-4, 0, 4], [-60, -54, -60], color='#3a3a3a', lw=1.5) # legs\n", " ax.plot([-8, 0, 5], [-45, -44, -42], color='#3a3a3a', lw=1.5) # arms\n", " ax.add_artist(patches.Circle((0, -38), 3, fc='none', ec='#3a3a3a', lw=1.5))\n", " ax.add_artist(patches.Arc((-9, -45), 7, 15, theta1=90, theta2=270, color='#3a3a3a', lw=1.5))\n", " if len(shots):\n", " ax.scatter([s[0] for s in shots], [s[1] for s in shots], s=42, linewidths=1.6,\n", " facecolors='none' if hollow else colour, edgecolors=colour, zorder=2)\n", " ax.set_xlim(-40, 40); ax.set_ylim(-64, 40); ax.set_aspect('equal')\n", " ax.set_xticks([]); ax.set_yticks([]); ax.grid(False)\n", " for side in ('top', 'right', 'bottom', 'left'):\n", " ax.spines[side].set_visible(False)\n", "\n", "rng = random.Random(4)\n", "first_ten = [shoot((0.0, 0.0), rng) for _ in range(10)]\n", "save_fig('the-range', lambda ax: draw_range(ax, first_ten, hollow=True), figsize=(5.0, 5.6))" ] }, { "cell_type": "markdown", "id": "c76cea74", "metadata": {}, "source": [ "## 2. Two robots\n", "\n", "**Stubborn** aims at the bullseye every time. It is not broken and it is not\n", "lazy — it simply has no way of knowing that anything is wrong.\n", "\n", "**Listens** is told, after each arrow, how far off it landed and in which\n", "direction, and shifts its aim by a fraction of that. Nothing else about it is\n", "different: same bow, same habit, same wobble." ] }, { "cell_type": "code", "execution_count": 4, "id": "d0604d07", "metadata": { "execution": { "iopub.execute_input": "2026-08-19T08:39:44.095037Z", "iopub.status.busy": "2026-08-19T08:39:44.094955Z", "iopub.status.idle": "2026-08-19T08:39:44.125375Z", "shell.execute_reply": "2026-08-19T08:39:44.125139Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "step = 0.15\n", "first arrow : 13.7 cm vs 13.7 cm\n", "last arrow : 13.5 cm vs 5.2 cm\n" ] } ], "source": [ "#| caption: The only difference is what happens after the arrow lands\n", "STEP = 0.15\n", "\n", "def stubborn(seed):\n", " rng = random.Random(seed)\n", " return [miss_by(shoot((0.0, 0.0), rng)) for _ in range(ARROWS)]\n", "\n", "def listens(seed):\n", " rng = random.Random(seed)\n", " aim, misses = [0.0, 0.0], []\n", " for _ in range(ARROWS):\n", " shot = shoot(tuple(aim), rng)\n", " misses.append(miss_by(shot))\n", " # The feedback: where it landed. Move BOTH parameters a fraction of\n", " # the way back — direction from the sideways miss, tension from the\n", " # up-and-down one. Learning is these two numbers changing.\n", " aim[0] -= STEP * shot[0]\n", " aim[1] -= STEP * shot[1]\n", " return misses\n", "\n", "record('step', STEP)\n", "\n", "def average_curve(robot):\n", " runs = [robot(s) for s in range(TRIALS)]\n", " return [sum(r[i] for r in runs) / TRIALS for i in range(ARROWS)]\n", "\n", "curve_stubborn = average_curve(stubborn)\n", "curve_listens = average_curve(listens)\n", "\n", "print('first arrow :', round(curve_stubborn[0], 1), 'cm vs', round(curve_listens[0], 1), 'cm')\n", "print('last arrow :', round(curve_stubborn[-1], 1), 'cm vs', round(curve_listens[-1], 1), 'cm')" ] }, { "cell_type": "markdown", "id": "5f2ee389", "metadata": {}, "source": [ "## 3. What learning looks like\n", "\n", "Two lines. One of them is learning and one of them is not, and there is no\n", "third thing you need to know to tell which is which." ] }, { "cell_type": "code", "execution_count": 5, "id": "24c4fdbf", "metadata": { "execution": { "iopub.execute_input": "2026-08-19T08:39:44.126480Z", "iopub.status.busy": "2026-08-19T08:39:44.126406Z", "iopub.status.idle": "2026-08-19T08:39:44.206064Z", "shell.execute_reply": "2026-08-19T08:39:44.205828Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "first_miss = 13.7\n", "last_miss = 5.2\n", "stubborn_last = 13.5\n", "floor = 5.2\n", "unbeatable = 5.0\n", "figure learning-curve -> the-robot-with-a-bow.learning-curve.{light,dark}.svg\n" ] } ], "source": [ "#| caption: Distance from the bullseye, arrow by arrow\n", "record('first_miss', round(curve_listens[0], 1))\n", "record('last_miss', round(curve_listens[-1], 1))\n", "record('stubborn_last', round(curve_stubborn[-1], 1))\n", "record('floor', round(sum(curve_listens[-10:]) / 10, 1))\n", "\n", "# The wobble alone, with no bias at all: the best any archer could ever do.\n", "rng = random.Random(99)\n", "perfect = sum(miss_by((rng.gauss(0, WOBBLE), rng.gauss(0, WOBBLE)))\n", " for _ in range(20000)) / 20000\n", "record('unbeatable', round(perfect, 1))\n", "\n", "def plot_curves(ax):\n", " x = range(1, ARROWS + 1)\n", " ax.plot(x, curve_stubborn, color='#9aa1ab', linewidth=2, label='told nothing')\n", " ax.plot(x, curve_listens, color='#ee785b', linewidth=2.4, label='told where it landed')\n", " ax.axhline(perfect, color='#6b7280', linestyle=':', linewidth=1.2)\n", " ax.text(ARROWS * 0.55, perfect + 0.8, 'the wobble — nothing can beat this',\n", " fontsize=9.5, color='#6b7280')\n", " ax.set_xlabel('arrows shot')\n", " ax.set_ylabel('distance from the bullseye (cm)')\n", " ax.set_ylim(0, max(curve_stubborn) * 1.15)\n", " ax.legend(frameon=False, fontsize=10)\n", " ax.spines[['top', 'right']].set_visible(False)\n", "\n", "save_fig('learning-curve', plot_curves, figsize=(7.4, 4.2))" ] }, { "cell_type": "markdown", "id": "c92c8d50", "metadata": {}, "source": [ "## 4. Where the arrows actually go\n", "\n", "The curve says *how far off*. The target says *where*, and that is the part\n", "that shows what was learned: the habit is gone, and the scatter is not." ] }, { "cell_type": "code", "execution_count": 6, "id": "5fcc5f0e", "metadata": { "execution": { "iopub.execute_input": "2026-08-19T08:39:44.207337Z", "iopub.status.busy": "2026-08-19T08:39:44.207210Z", "iopub.status.idle": "2026-08-19T08:39:44.247982Z", "shell.execute_reply": "2026-08-19T08:39:44.247754Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "early_in_gold = 5\n", "late_in_gold = 10\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "figure on-the-target -> the-robot-with-a-bow.on-the-target.{light,dark}.svg\n" ] } ], "source": [ "#| caption: The first ten arrows and the last ten, on the same range\n", "rng = random.Random(7)\n", "aim, shots = [0.0, 0.0], []\n", "for _ in range(ARROWS):\n", " shot = shoot(tuple(aim), rng)\n", " shots.append(shot)\n", " aim[0] -= STEP * shot[0]\n", " aim[1] -= STEP * shot[1]\n", "\n", "early, late = shots[:10], shots[-10:]\n", "record('early_in_gold', sum(1 for s in early if miss_by(s) <= GOLD))\n", "record('late_in_gold', sum(1 for s in late if miss_by(s) <= GOLD))\n", "\n", "def plot_target(ax):\n", " draw_range(ax, early, colour='#9aa1ab', hollow=True)\n", " ax.scatter([s[0] for s in late], [s[1] for s in late], s=42, color='#ee785b', zorder=3)\n", " ax.legend(handles=[\n", " plt.Line2D([], [], marker='o', ls='', mfc='none', mec='#9aa1ab', label='first ten arrows'),\n", " plt.Line2D([], [], marker='o', ls='', color='#ee785b', label='last ten arrows'),\n", " ], frameon=False, fontsize=10, loc='lower right')\n", "\n", "save_fig('on-the-target', plot_target, figsize=(5.0, 5.6))" ] }, { "cell_type": "markdown", "id": "8f7f9de7", "metadata": {}, "source": [ "## 5. What experience is worth, and when it stops being worth it\n", "\n", "The curve above falls fastest at the start. That is not a detail of this\n", "simulation — it is the shape of nearly every learning curve you will ever\n", "meet, and it is why the tenth example is worth more than the thousandth." ] }, { "cell_type": "code", "execution_count": 7, "id": "064d1d47", "metadata": { "execution": { "iopub.execute_input": "2026-08-19T08:39:44.249165Z", "iopub.status.busy": "2026-08-19T08:39:44.249086Z", "iopub.status.idle": "2026-08-19T08:39:44.302008Z", "shell.execute_reply": "2026-08-19T08:39:44.301413Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gain_first = 3.2\n", "gain_last = 0.0\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "figure what-each-block-bought -> the-robot-with-a-bow.what-each-block-bought.{light,dark}.svg\n", "gains per block of ten: [3.2, 0.2, -0.1, 0.1, 0.0]\n" ] } ], "source": [ "#| caption: How much each block of ten arrows bought\n", "blocks = [(i, sum(curve_listens[i:i + 10]) / 10) for i in range(0, ARROWS, 10)]\n", "def gain(i):\n", " g = round(blocks[i - 1][1] - blocks[i][1], 1)\n", " return 0.0 if g == 0 else g # a block that bought nothing is 0.0, not -0.0\n", "\n", "gains = [gain(i) for i in range(1, len(blocks))]\n", "record('gain_first', gains[0])\n", "record('gain_last', gains[-1])\n", "\n", "def plot_gains(ax):\n", " labels = [f'{b[0] + 1}–{b[0] + 10}' for b in blocks[1:]]\n", " bars = ax.bar(labels, gains, color=['#ee785b'] + ['#9aa1ab'] * (len(gains) - 1))\n", " for b, v in zip(bars, gains):\n", " ax.text(b.get_x() + b.get_width() / 2, v + 0.12, f'{v}', ha='center', fontsize=10)\n", " ax.set_ylabel('cm closer than the block before')\n", " ax.set_xlabel('arrows')\n", " ax.spines[['top', 'right']].set_visible(False)\n", "\n", "save_fig('what-each-block-bought', plot_gains, figsize=(7.4, 3.8))\n", "print('gains per block of ten:', gains)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.5" } }, "nbformat": 4, "nbformat_minor": 5 }