{ "cells": [ { "cell_type": "markdown", "id": "33ca0e05", "metadata": {}, "source": [ "# How a Model is Fitted · L2 — The cost is a choice\n", "\n", "`a1-regression` scored a line by adding up its squared errors and never said\n", "why squared. This notebook tries the obvious alternative — add the sizes of the\n", "gaps without squaring them — on the same 200 markets.\n", "\n", "**They pick different lines.** Each cost ranks its own line first, so nothing\n", "outside the two settles it. Then one odd market shows what the choice actually\n", "buys: the squared line moves fifteen times as far as the absolute one.\n", "\n", "Every figure and number this notebook produces is what the lesson prints.\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "d48c6fa4", "metadata": { "execution": { "iopub.execute_input": "2026-08-20T13:46:43.735831Z", "iopub.status.busy": "2026-08-20T13:46:43.735689Z", "iopub.status.idle": "2026-08-20T13:46:44.166226Z", "shell.execute_reply": "2026-08-20T13:46:44.165938Z" } }, "outputs": [], "source": [ "# hide — plumbing so this file runs both in the repo and in Colab.\n", "try:\n", " from _figkit import save_fig, record\n", "except ImportError: # Colab — no repo, no problem\n", " def save_fig(name, plot, **kw):\n", " import matplotlib.pyplot as plt\n", " fig, ax = plt.subplots(figsize=kw.get('figsize', (7, 4.2))); plot(ax); plt.show()\n", " def record(key, value):\n", " print(f'{key} = {value}'); return value\n", "\n", "from pathlib import Path\n", "REL = 'public/datasets/a1-regression/advertising-media-mix.csv'\n", "here = Path.cwd()\n", "LOCAL = next((p / REL for p in [here, *here.parents] if (p / REL).exists()), None)\n", "CSV = LOCAL or 'https://raw.githubusercontent.com/Dr-Shashank-S-Sharma/expedify-ai-courses/main/datasets/a1-regression/advertising-media-mix.csv'\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "d13e4234", "metadata": { "execution": { "iopub.execute_input": "2026-08-20T13:46:44.167499Z", "iopub.status.busy": "2026-08-20T13:46:44.167404Z", "iopub.status.idle": "2026-08-20T13:46:45.924910Z", "shell.execute_reply": "2026-08-20T13:46:45.924686Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "n_markets = 200\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
tv_spendsales
0230.122.1
144.510.4
217.29.3
3151.518.5
4180.812.9
\n", "
" ], "text/plain": [ " tv_spend sales\n", "0 230.1 22.1\n", "1 44.5 10.4\n", "2 17.2 9.3\n", "3 151.5 18.5\n", "4 180.8 12.9" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#| caption: The same 200 markets a1-regression used\n", "import numpy as np\n", "import pandas as pd\n", "from scipy.optimize import minimize\n", "\n", "df = pd.read_csv(CSV)\n", "x = df['tv_spend'].to_numpy() # TV budget, $ thousands\n", "y = df['sales'].to_numpy() # sales, thousands of units\n", "record('n_markets', len(y))\n", "\n", "def mse(a, b, X=x, Y=y):\n", " return float(np.mean((Y - (a + b * X)) ** 2))\n", "\n", "def mae(a, b, X=x, Y=y):\n", " return float(np.mean(np.abs(Y - (a + b * X))))\n", "\n", "df[['tv_spend', 'sales']].head()\n" ] }, { "cell_type": "markdown", "id": "40b0dccb", "metadata": {}, "source": [ "## 1. The cost is built from the gaps\n", "\n", "A line predicts a sales figure for every market. The market sold something\n", "else. The difference is the gap — the *residual* — and the cost is built from\n", "all 200 of them.\n" ] }, { "cell_type": "code", "execution_count": 3, "id": "6c76ab68", "metadata": { "execution": { "iopub.execute_input": "2026-08-20T13:46:45.926019Z", "iopub.status.busy": "2026-08-20T13:46:45.925944Z", "iopub.status.idle": "2026-08-20T13:46:46.047142Z", "shell.execute_reply": "2026-08-20T13:46:46.046821Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "sq_intercept = 7.033\n", "sq_slope = 0.04754\n", "sq_mse = 10.513\n", "sq_mae = 2.55\n", "biggest_gap = 8.39\n", "typical_gap = 2.03\n", "figure the-gaps -> the-cost-is-a-choice.the-gaps.{light,dark}.svg\n" ] } ], "source": [ "#| caption: The gaps a cost is built from\n", "b_sq, a_sq = np.polyfit(x, y, 1)\n", "record('sq_intercept', round(a_sq, 3))\n", "record('sq_slope', round(b_sq, 5))\n", "record('sq_mse', round(mse(a_sq, b_sq), 3))\n", "record('sq_mae', round(mae(a_sq, b_sq), 3))\n", "\n", "resid = y - (a_sq + b_sq * x)\n", "record('biggest_gap', round(float(np.max(np.abs(resid))), 2))\n", "record('typical_gap', round(float(np.median(np.abs(resid))), 2))\n", "\n", "def plot_gaps(ax):\n", " order = np.argsort(x)\n", " ax.scatter(x, y, s=14, color='#9aa0aa', zorder=3)\n", " for xi, yi in zip(x, y):\n", " ax.plot([xi, xi], [yi, a_sq + b_sq * xi], color='#e2574c', lw=0.7, alpha=0.55, zorder=2)\n", " ax.plot(x[order], (a_sq + b_sq * x)[order], color='#3b6fd4', lw=2, zorder=4)\n", " ax.set_xlabel('TV budget ($ thousands)')\n", " ax.set_ylabel('Sales (thousands of units)')\n", "\n", "save_fig('the-gaps', plot_gaps)\n" ] }, { "cell_type": "markdown", "id": "7e63b014", "metadata": {}, "source": [ "## 2. Two honest ways to turn 200 gaps into one number\n", "\n", "Square each gap and average them, or take the size of each gap and average\n", "those. Both give one number, both are smallest when the line sits in the middle\n", "of the cloud, and both are used in practice.\n", "\n", "The second is fitted by search rather than by formula — there is no\n", "least-squares shortcut for it.\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "2619f8bb", "metadata": { "execution": { "iopub.execute_input": "2026-08-20T13:46:46.048432Z", "iopub.status.busy": "2026-08-20T13:46:46.048359Z", "iopub.status.idle": "2026-08-20T13:46:46.054515Z", "shell.execute_reply": "2026-08-20T13:46:46.054268Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "abs_intercept = 6.529\n", "abs_slope = 0.05063\n", "abs_mse = 10.585\n", "abs_mae = 2.534\n", "cross_columns = ['', 'Squared error', 'Absolute error']\n", "cross_rows = [['The line squared error picks (slope 0.0475)', '10.513', '2.550'], ['The line absolute error picks (slope 0.0506)', '10.585', '2.534']]\n", "squared picks slope 0.04754, intercept 7.033\n", "absolute picks slope 0.05063, intercept 6.529\n" ] } ], "source": [ "#| caption: Fit the same straight line under each cost\n", "best_abs = minimize(lambda p: mae(p[0], p[1]), [a_sq, b_sq], method='Nelder-Mead',\n", " options=dict(xatol=1e-9, fatol=1e-11, maxiter=50000))\n", "a_ab, b_ab = best_abs.x\n", "record('abs_intercept', round(a_ab, 3))\n", "record('abs_slope', round(b_ab, 5))\n", "record('abs_mse', round(mse(a_ab, b_ab), 3))\n", "record('abs_mae', round(mae(a_ab, b_ab), 3))\n", "\n", "# The cross-table the lesson prints: each line scored under BOTH costs.\n", "record('cross_columns', ['', 'Squared error', 'Absolute error'])\n", "record('cross_rows', [\n", " [f'The line squared error picks (slope {b_sq:.4f})', f'{mse(a_sq, b_sq):.3f}', f'{mae(a_sq, b_sq):.3f}'],\n", " [f'The line absolute error picks (slope {b_ab:.4f})', f'{mse(a_ab, b_ab):.3f}', f'{mae(a_ab, b_ab):.3f}'],\n", "])\n", "print(f'squared picks slope {b_sq:.5f}, intercept {a_sq:.3f}')\n", "print(f'absolute picks slope {b_ab:.5f}, intercept {a_ab:.3f}')\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "4be30c83", "metadata": { "execution": { "iopub.execute_input": "2026-08-20T13:46:46.055735Z", "iopub.status.busy": "2026-08-20T13:46:46.055651Z", "iopub.status.idle": "2026-08-20T13:46:46.120033Z", "shell.execute_reply": "2026-08-20T13:46:46.119736Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "figure two-costs-two-lines -> the-cost-is-a-choice.two-costs-two-lines.{light,dark}.svg\n" ] } ], "source": [ "#| caption: Two costs, two lines\n", "def plot_two(ax):\n", " grid = np.linspace(x.min(), x.max(), 200)\n", " ax.scatter(x, y, s=14, color='#c9c9d1', zorder=2)\n", " ax.plot(grid, a_sq + b_sq * grid, color='#e2574c', lw=2.2,\n", " label=f'squared error (slope {b_sq:.4f})', zorder=4)\n", " ax.plot(grid, a_ab + b_ab * grid, color='#3b6fd4', lw=2.2, ls='--',\n", " label=f'absolute error (slope {b_ab:.4f})', zorder=3)\n", " ax.set_xlabel('TV budget ($ thousands)')\n", " ax.set_ylabel('Sales (thousands of units)')\n", " ax.legend(frameon=False, loc='upper left')\n", "\n", "save_fig('two-costs-two-lines', plot_two)\n" ] }, { "cell_type": "markdown", "id": "4cc8cfb0", "metadata": {}, "source": [ "## 3. One market that spent almost nothing and sold a lot\n", "\n", "Add a single market to the file: a $10,000 TV budget and 26,000 units sold.\n", "Nothing else changes. Refit both lines and measure how far each one moved.\n" ] }, { "cell_type": "code", "execution_count": 6, "id": "1663904d", "metadata": { "execution": { "iopub.execute_input": "2026-08-20T13:46:46.121671Z", "iopub.status.busy": "2026-08-20T13:46:46.121561Z", "iopub.status.idle": "2026-08-20T13:46:46.130455Z", "shell.execute_reply": "2026-08-20T13:46:46.130171Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "outlier_tv = 10.0\n", "outlier_sales = 26.0\n", "sq_slope_after = 0.04584\n", "abs_slope_after = 0.05052\n", "sq_moved = 0.00170\n", "abs_moved = 0.00011\n", "moved_ratio = 16\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "outlier_gap = 18.5\n", "outlier_gap_squared = 342\n", "outlier_columns = ['', 'Slope before', 'Slope after', 'Moved by']\n", "outlier_rows = [['Squared error', '0.0475', '0.0458', '0.0017'], ['Absolute error', '0.0506', '0.0505', '0.0001']]\n", "squared moved 0.00170, absolute moved 0.00011 — 15.7x\n" ] } ], "source": [ "#| caption: What one odd market does to each line\n", "OUT_TV, OUT_SALES = 10.0, 26.0\n", "record('outlier_tv', OUT_TV)\n", "record('outlier_sales', OUT_SALES)\n", "\n", "X2, Y2 = np.append(x, OUT_TV), np.append(y, OUT_SALES)\n", "b_sq2, a_sq2 = np.polyfit(X2, Y2, 1)\n", "r2 = minimize(lambda p: mae(p[0], p[1], X2, Y2), [a_ab, b_ab], method='Nelder-Mead',\n", " options=dict(xatol=1e-9, fatol=1e-11, maxiter=50000))\n", "a_ab2, b_ab2 = r2.x\n", "\n", "moved_sq, moved_ab = abs(b_sq2 - b_sq), abs(b_ab2 - b_ab)\n", "record('sq_slope_after', round(b_sq2, 5))\n", "record('abs_slope_after', round(b_ab2, 5))\n", "record('sq_moved', f'{moved_sq:.5f}')\n", "record('abs_moved', f'{moved_ab:.5f}')\n", "record('moved_ratio', round(moved_sq / moved_ab))\n", "record('outlier_gap', round(float(abs(OUT_SALES - (a_sq + b_sq * OUT_TV))), 1))\n", "record('outlier_gap_squared', round(float((OUT_SALES - (a_sq + b_sq * OUT_TV)) ** 2)))\n", "\n", "record('outlier_columns', ['', 'Slope before', 'Slope after', 'Moved by'])\n", "record('outlier_rows', [\n", " ['Squared error', f'{b_sq:.4f}', f'{b_sq2:.4f}', f'{moved_sq:.4f}'],\n", " ['Absolute error', f'{b_ab:.4f}', f'{b_ab2:.4f}', f'{moved_ab:.4f}'],\n", "])\n", "print(f'squared moved {moved_sq:.5f}, absolute moved {moved_ab:.5f} — {moved_sq/moved_ab:.1f}x')\n" ] }, { "cell_type": "code", "execution_count": 7, "id": "5efc52e5", "metadata": { "execution": { "iopub.execute_input": "2026-08-20T13:46:46.131936Z", "iopub.status.busy": "2026-08-20T13:46:46.131826Z", "iopub.status.idle": "2026-08-20T13:46:46.198512Z", "shell.execute_reply": "2026-08-20T13:46:46.198298Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "figure what-one-outlier-does -> the-cost-is-a-choice.what-one-outlier-does.{light,dark}.svg\n" ] } ], "source": [ "#| caption: The same market, added to both\n", "def plot_outlier(ax):\n", " grid = np.linspace(0, x.max(), 200)\n", " ax.scatter(x, y, s=13, color='#d5d5db', zorder=2)\n", " ax.scatter([OUT_TV], [OUT_SALES], s=70, color='#c2871a', zorder=6, marker='D')\n", " ax.annotate('the added market', (OUT_TV, OUT_SALES), textcoords='offset points',\n", " xytext=(12, -2), fontsize=9, color='#77777f')\n", " ax.plot(grid, a_sq + b_sq * grid, color='#e2574c', lw=1.2, alpha=0.45, zorder=3)\n", " ax.plot(grid, a_sq2 + b_sq2 * grid, color='#e2574c', lw=2.2, zorder=5,\n", " label=f'squared error, moved {moved_sq:.4f}')\n", " ax.plot(grid, a_ab + b_ab * grid, color='#3b6fd4', lw=1.2, alpha=0.45, ls='--', zorder=3)\n", " ax.plot(grid, a_ab2 + b_ab2 * grid, color='#3b6fd4', lw=2.2, ls='--', zorder=4,\n", " label=f'absolute error, moved {moved_ab:.4f}')\n", " ax.set_xlabel('TV budget ($ thousands)')\n", " ax.set_ylabel('Sales (thousands of units)')\n", " ax.legend(frameon=False, loc='lower right')\n", "\n", "save_fig('what-one-outlier-does', plot_outlier)\n" ] }, { "cell_type": "markdown", "id": "527f8030", "metadata": {}, "source": [ "## What this notebook establishes\n", "\n", "- Two costs, both honest, pick **different lines** from the same 200 markets.\n", "- Each cost ranks its own line first, so the two cannot settle it between them.\n", "- Squaring makes one large gap count enormously, so a single market moves the\n", " squared line much further than the absolute one.\n", "\n", "**Worth changing:** move the added market's sales figure, or its TV budget, and\n", "watch which line notices. A market far out along the x-axis moves both lines;\n", "one far out in y moves only the squared one.\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.5" } }, "nbformat": 4, "nbformat_minor": 5 }