Skip to content
Expedify
14 min

A* and heuristic search — search you can afford

A machine plays chess better than anybody, and it never sees the end of the game. Two ideas separate it from the machine that solved tic-tac-toe, and neither of them is a faster computer.

The machine that looked ahead ended at a wall. Walking the chess tree would take longer than the universe has existed.

And yet a machine plays chess better than anybody reading this. So it is not doing what you just watched.

Two ideas separate the two machines, and neither of them is a faster computer.

Breadth-first search examined 1,207 of the 1,344 cells

Here is a city with walls in it: 1,344 cells, 136 of them blocked. Start at the bottom-left corner, finish at the top-right.

The obvious method is last lesson's. Expand the nearest unexplored cell, over and over, in every direction at once. It is called breadth-first search, and it always finds the shortest route.

Breadth-first search. Shaded means examined.introduction-to-ai/search-you-can-afford.ipynb

It examined 1,207 cells out of 1,344. It searched almost everywhere, including large regions in completely the wrong direction, because nothing in the method knows where the destination is.

Add a guess at the distance still to come

Change one thing. When choosing which cell to expand next, prefer the one with the best total: the distance already walked, plus a guess at the distance left.

f(n)  =  g(n)steps taken so far  +  h(n)guess at steps remainingf(n) \;=\; \underbrace{g(n)}_{\text{steps taken so far}} \;+\; \underbrace{h(n)}_{\text{guess at steps remaining}}
The whole of A*. One line, and the only new thing in it is h — the guess.

The guess here is the simplest one available: how far away the goal is if you ignore every wall. It is always a little optimistic, and that turns out to be the property that matters.

A* examined 124 cells instead of 1,207

A* on the identical problem.introduction-to-ai/search-you-can-afford.ipynb

124 cells instead of 1,207. That is 90%% of the work never done, about 9.7 times less. And the route it returns is the same length, which the notebook checks.

The same route, found either way.introduction-to-ai/search-you-can-afford.ipynb

Compare those two numbers with each other, not with the size of the map. A* examined 124 cells to find a route 75 cells long. It barely wandered at all, and that is what a good guess buys.

A guess that never overestimates can never talk you out of the best route

That is the property the straight-line distance has. It ignores walls, so it can only ever be too optimistic. An optimistic guess keeps a promising route on the list until it has been checked.

It is fussier than it looks. The first version of this notebook saved almost nothing. On a grid where every step costs the same, huge numbers of cells score an identical total. A* then has no reason to prefer any of them.

Stopping two moves early costs 6 games in 400 and saves 2,000 times the work

A* still walks all the way to its goal. Chess cannot, because there is no reaching the end. So the second idea is blunter: look ahead a fixed number of moves, then stop and estimate how good the position is.

That estimate is where the chess knowledge lives. Material, king safety, control of the centre. It is a human's judgement written down as arithmetic.

Does giving up the guarantee cost anything? Back to tic-tac-toe, where the perfect answer is known. Two players, four hundred games each against a random opponent.

The same opponent and the same number of games.

all the way to the end

Games lost, of 400
0
Positions examined per game
309,898

two moves, then guess

Games lost, of 400
6
Positions examined per game
135

A guarantee exchanged for a rounding error, at a colossal discount. That is the deal chess engines take, and it is the first honest cost-benefit calculation in this course.

Somebody sat down and invented the guess

Both ideas rest on the same thing. A person decided that straight-line distance was a sensible estimate for a city. A person decided what a bishop is worth.

Which puts this in the top half of Five ways to build it's spectrum. A human supplies the knowledge, and the machine executes it faster and more tirelessly than any human could.

So if the quality of the search depends on the quality of the guess, could a machine learn the guess? It can. That is what happened to Go in 2016, and it is why a program made moves professionals called beautiful and could not explain.

But before a machine can learn a guess, there has to be a way for a machine to learn anything. That is where this path goes next.

The city, both searches, and what a guess is worth

content/notebooks/introduction-to-ai/search-you-can-afford.ipynb

Move the walls, or change the guess. Worth trying: multiply the guess by 1.5 so it sometimes overestimates, and check whether the route it returns is still the shortest.

Show the code6 cells
Build the map
W, H = 48, 28
rng = random.Random(11)
grid = [[0]*W for _ in range(H)]
for _ in range(16):                       # sparse walls, with gaps to walk through
    if rng.random() < .5:
        r = rng.randrange(H); c0 = rng.randrange(W-13)
        for c in range(c0, c0+13):
            if rng.random() > .22: grid[r][c] = 1
    else:
        c = rng.randrange(W); r0 = rng.randrange(max(1, H-10))
        for r in range(r0, min(H, r0+10)):
            if rng.random() > .22: grid[r][c] = 1
START, GOAL = (H-1, 0), (0, W-1)
grid[START[0]][START[1]] = grid[GOAL[0]][GOAL[1]] = 0

def neighbours(rc):
    r, c = rc
    for dr, dc in ((1,0),(-1,0),(0,1),(0,-1)):
        nr, nc = r+dr, c+dc
        if 0 <= nr < H and 0 <= nc < W and grid[nr][nc] == 0:
            yield (nr, nc)
record('map_cells', f'{W*H:,}')
record('map_walls', f'{sum(sum(r) for r in grid):,}')
Breadth-first search
def bfs():
    seen = {START: None}; q = deque([START]); order = []
    while q:
        cur = q.popleft(); order.append(cur)
        if cur == GOAL: break
        for n in neighbours(cur):
            if n not in seen:
                seen[n] = cur; q.append(n)
    path, cur = [], GOAL
    while cur: path.append(cur); cur = seen.get(cur)
    return order, path[::-1]

bfs_order, bfs_path = bfs()
record('bfs_explored', f'{len(bfs_order):,}')
record('bfs_path_len', f'{len(bfs_path):,}')
A* with a straight-line guess
def astar():
    def h(rc): return abs(rc[0]-GOAL[0]) + abs(rc[1]-GOAL[1])
    seen = {START: None}; g = {START: 0}; order = []
    # (total guess, then how far is left) — the second term is a TIE-BREAK.
    # Without it, every monotone route across an open grid scores the same
    # total, A* has no reason to prefer any of them, and it degenerates
    # into breadth-first. Prefer the cell closest to the goal and it commits.
    pq = [(h(START), h(START), 0, START)]
    while pq:
        _, _, gc, cur = heapq.heappop(pq)
        order.append(cur)
        if cur == GOAL: break
        for n in neighbours(cur):
            ng = gc + 1
            if ng < g.get(n, 10**9):
                g[n] = ng; seen[n] = cur
                heapq.heappush(pq, (ng + h(n), h(n), ng, n))
    path, cur = [], GOAL
    while cur: path.append(cur); cur = seen.get(cur)
    return order, path[::-1]

ast_order, ast_path = astar()
record('astar_explored', f'{len(ast_order):,}')
record('astar_path_len', f'{len(ast_path):,}')
record('same_length', 'yes' if len(ast_path) == len(bfs_path) else 'NO')
record('astar_factor', f'{len(bfs_order)/len(ast_order):.1f}')
record('astar_saved_pct', f'{100*(1-len(ast_order)/len(bfs_order)):.0f}%')
What breadth-first looked at
# Rectangles, not imshow. imshow embeds the grid as a base64 PNG inside the
# SVG, and lib/blocks/svg.ts refuses any <image> — so the figure would have
# been replaced by its alt text on the page. The test suite caught it.
#
# Adjacent cells in a row are then merged into ONE rectangle. Drawing 1,200
# separate squares produced a 254 KB SVG, which is inlined into the page
# twice (light and dark); runs bring it under 30 KB and look identical.
from matplotlib.patches import Rectangle
from matplotlib.collections import PatchCollection

WALL     = (0.55, 0.53, 0.50, 0.55)
EXPLORED = (0.91, 0.41, 0.25, 0.30)
PATH     = (0.91, 0.41, 0.25, 1.00)

def runs(coords):
    """(row, col) cells -> (row, col_start, width) horizontal runs."""
    by_row = {}
    for r, c in coords: by_row.setdefault(r, []).append(c)
    out = []
    for r, cs in by_row.items():
        cs.sort(); start = prev = cs[0]
        for c in cs[1:]:
            if c == prev + 1: prev = c; continue
            out.append((r, start, prev - start + 1)); start = prev = c
        out.append((r, start, prev - start + 1))
    return out

def cells(coords, colour, ax):
    if not coords: return
    ax.add_collection(PatchCollection(
        [Rectangle((c, r), w, 1) for r, c, w in runs(coords)],
        facecolor=colour, edgecolor='none'))

def draw(order, path, ax):
    walls = [(r, c) for r in range(H) for c in range(W) if grid[r][c]]
    seen  = [(r, c) for (r, c) in order if not grid[r][c]]
    cells(seen, EXPLORED, ax)
    cells(walls, WALL, ax)
    cells(path, PATH, ax)
    ax.set_xlim(0, W); ax.set_ylim(H, 0)
    ax.set_aspect('equal'); ax.set_xticks([]); ax.set_yticks([])
    for s in ax.spines.values(): s.set_visible(False)
    record('draw_runs_' + ('bfs' if len(order) > 500 else 'astar'), f'{len(runs(seen)):,}')

save_fig('flood-bfs', lambda ax: draw(bfs_order, bfs_path, ax), figsize=(7, 4.2))
What A* looked at, on the same map
save_fig('beam-astar', lambda ax: draw(ast_order, ast_path, ax), figsize=(7, 4.0))

def plot(ax):
    bars = ax.bar(['look everywhere\n(breadth-first)', 'look towards the goal\n(A*)'],
                  [len(bfs_order), len(ast_order)])
    bars[1].set_alpha(.55)
    for b_, v in zip(bars, [len(bfs_order), len(ast_order)]):
        ax.text(b_.get_x()+b_.get_width()/2, v*1.04, f'{v:,}', ha='center', fontsize=10)
    ax.set_ylabel('cells examined')
    ax.set_ylim(0, len(bfs_order)*1.18)
save_fig('cells-compared', plot, figsize=(7, 3.4))
Depth-limited search with a crude estimate
LINES = [(0,1,2),(3,4,5),(6,7,8),(0,3,6),(1,4,7),(2,5,8),(0,4,8),(2,4,6)]
EMPTY = (None,)*9
def winner(b):
    for i,j,k in LINES:
        if b[i] is not None and b[i]==b[j]==b[k]: return b[i]
    return None
def moves(b): return [i for i,c in enumerate(b) if c is None]
def play(b,i,m): return b[:i]+(m,)+b[i+1:]

def guess(b, me):                       # a two-line evaluation function
    them = 'O' if me=='X' else 'X'
    s = 0
    for i,j,k in LINES:
        line = [b[i],b[j],b[k]]
        if them not in line: s += line.count(me)
        if me   not in line: s -= line.count(them)
    return s / 24

def limited(b, mark, me, depth, counter):
    counter[0] += 1
    w = winner(b)
    if w: return 1.0 if w==me else -1.0
    if not moves(b): return 0.0
    if depth == 0: return guess(b, me)
    vals = [limited(play(b,m,mark), 'O' if mark=='X' else 'X', me, depth-1, counter)
            for m in moves(b)]
    return max(vals) if mark==me else min(vals)

def move_limited(b, mark, depth, counter):
    scored = [(limited(play(b,m,mark),'O' if mark=='X' else 'X',mark,depth-1,counter), m)
              for m in moves(b)]
    return max(scored)[1]

rng2 = random.Random(5)
for depth, key in ((2,'d2'), (9,'d9')):
    res = {'win':0,'draw':0,'loss':0}; c = [0]
    for game in range(400):
        ai = 'X' if game % 2 == 0 else 'O'
        b, mark = EMPTY, 'X'
        while winner(b) is None and moves(b):
            m = move_limited(b, mark, depth, c) if mark == ai else rng2.choice(moves(b))
            b = play(b, m, mark); mark = 'O' if mark=='X' else 'X'
        w = winner(b)
        res['draw' if w is None else 'win' if w==ai else 'loss'] += 1
    record(f'{key}_losses', f"{res['loss']:,}")
    record(f'{key}_wins', f"{res['win']:,}")
    record(f'{key}_nodes', f'{c[0]:,}')
    record(f'{key}_nodes_per_game', f'{c[0]/400:,.0f}')

Related lessons