ARTIFICIAL INTELLIGENCE
AlphaGo’s Move 37: search, learning, and a surprising move
Move 37 in AlphaGo’s second game against Lee Sedol revealed how learned preferences and search can diverge. The 2016 match also defines the limits of the claim.
Research checked
Newest cited publication
A Go board photographed at a tournament in Toulouse in November 2022 provides visual context for the game. AlphaGo’s Move 37 occurred in the separate 2016 match analyzed in the article.
Culture Japon / Wikimedia Commons. Commons-generated 1280-pixel thumbnail; no editorial changes. · Image source · CC BY-SA 4.0
On March 10, 2016, during the second game of its match against Lee Sedol in Seoul, AlphaGo played the move now remembered as Move 37. Its unfamiliar choice became an emblem of the encounter: a computer could produce strong play that surprised experienced observers. The moment is more interesting when explained through the system's machinery than when treated as an inexplicable flash of machine inspiration. AlphaGo combined learned move preferences, estimates of position strength, and selective search. That combination could favor an action that looked strange through the lens of familiar human play. It did not examine every possible future, prove that Go had been solved, or settle a general philosophical definition of creativity.
The January paper and the March match are separate events
The Nature paper describing AlphaGo appeared in January 2016, before the Lee Sedol series. It reported a five-to-zero victory over European champion Fan Hui in October 2015. Google's contemporaneous match report records a different result in Seoul: AlphaGo beat Lee Sedol four games to one in March 2016. Its second-game entry is dated March 10. The series used Chinese rules, a 7.5-point komi, and two hours of main time per player with specified overtime periods. These conditions define the achievement more precisely than a statement that a machine defeated humanity.
The distinction also prevents a common error in accounts of Move 37. The move belongs to the second game against Lee, not to the earlier five-game sweep against Fan. Lee's victory in the fourth game matters to the historical record because it demonstrates that the winning system was not infallible in that match. A final series score combines outcomes against one opponent under specified conditions. It is compelling evidence of playing strength, while remaining different from a mathematical guarantee that every chosen move was optimal. The January paper describes the earlier published system, rather than a released trace of the exact computation that chose Move 37.
Learning what experts play gives a useful starting point
The 2016 paper describes training a policy network on expert human moves, improving a related policy through self-play, and learning a separate value network from self-play outcomes. A policy assigns relative preference to available actions; a value estimate concerns the outlook from a position. Those roles differ. Predicting what an expert is likely to play is a useful source of candidates, but it does not make the most familiar candidate automatically the strongest move after further investigation.
A miniature example makes the distinction visible. Suppose a learned policy assigns move A a probability of 0.70 and move B a probability of 0.02 in a simplified position. Those numbers describe the model's action preferences before additional search. They are not probabilities that A and B are objectively correct. The model might have seen many similar examples favoring A while lacking examples where B works. Search can spend computation exploring consequences that the initial preference compresses imperfectly. This example illustrates the division of labor; its numbers are not reconstructed statistics for the real Move 37.
Search investigates promising futures without enumerating them all
AlphaGo used Monte Carlo tree search, which builds and updates a selectively explored tree of continuations. Its published procedure combined policy-guided exploration with evaluations using a value network and simulated continuations. The final move was chosen using accumulated search visits rather than simply taking the highest initial policy probability. Learned guidance made the search more useful, and search could in turn revise the practical ranking suggested by the learned guidance.
Consider a toy tree in which each position has ten possible replies. Looking exactly six moves ahead without pruning would expose one million distinct sequences at that depth alone. A selective procedure allocates effort unevenly: a promising branch receives more visits, while an uncertain branch may receive exploratory attention. This avoids the requirement to inspect all million sequences, but the saving has a cost. A decisive reply can remain unexplored or undervalued. The quality of the learned estimates and the available computation both matter. A strong search result is consequently evidence produced under a budget, not a proof that no better line exists.
Winning probability is not the same as immediate gain
A player's local gain can conflict with the longer-term position. Securing a few points now might leave weak groups elsewhere, while a move that looks unproductive locally could support later play. This general tension helps explain why an unfamiliar placement deserves investigation rather than immediate dismissal. It does not, by itself, reconstruct the tactical reasoning behind Move 37. That would require examining the actual board and subsequent variations, and it would still leave uncertainty about unplayed alternatives.
For example, imagine estimated winning chances of 0.65 after one candidate and 0.58 after another. A system optimizing the chance of victory would prefer the former under those estimates, even if the latter appeared likely to win by more points when it succeeded. If outcomes are encoded as plus one for a win and minus one for a loss, a winning probability of 0.65 corresponds to an expected outcome of 0.30: 0.65 minus 0.35. The calculation is exact for the toy assumptions. The estimates themselves could be inaccurate, which is why probability scores must not be confused with guarantees.
An unexpected good move is evidence with a limited scope
DeepMind's account identifies Move 37 in game two as an inventive choice that helped AlphaGo win. The memorable feature was its departure from familiar expectations, rather than a new rule or a change in the game's objective. An outcome can be novel relative to a player's experience while emerging from a completely specified computational procedure. Conversely, a surprising move can simply be bad. Surprise and quality are independent questions until the consequences are investigated.
The game supplies evidence that the system could produce useful departures from familiar play. It cannot establish, from that one choice alone, a complete causal account of the victory. Nor does it reveal whether the system had a human-like experience of imagination. A productive technical question is whether the same training-and-search method repeatedly finds effective unfamiliar strategies on fresh positions. A philosophical question asks which of those behaviors deserve the word creativity. Keeping the questions distinct allows the historical observation to remain powerful without turning it into proof of an undefined, universal mental faculty.
AlphaGo Zero should not be projected backward into 2016
In October 2017, DeepMind described AlphaGo Zero, a later system that learned through self-play without first training on human game examples. Its description also distinguished a combined policy-and-value network and the absence of the earlier rollout mechanism. Those changes belong to the later research result. They should not be retroactively used to claim that the system playing Move 37 had learned entirely without human games. Historical versions matter because they embody different experiments about how useful behavior can be learned.
Learning without human example games still requires a designed problem. The rules, legal-action interface, reward, network structure, search procedure, and training setup do not create themselves. Self-play can generate additional experience inside that framework. Calling it learning from scratch is meaningful with respect to omitted human demonstrations, but misleading if interpreted as an absence of assumptions or engineering. The progression from the earlier system to Zero is interesting precisely because a research team changed particular components and tested the consequences, rather than demonstrating a mysterious removal of all prior structure.
The game’s clarity was a major experimental advantage
Go offers a fully specified action space, observable board, and an eventual outcome under known rules. Many practical planning problems lack those conveniences. A robot may not observe all relevant physical state; a scientific simulator may be approximate; a business objective may change when people respond to a decision. Searching longer within an incorrect model can strengthen confidence in an unreliable answer. Transferring the general pattern of learned guidance plus search therefore requires evaluating the environment model and objective as carefully as the planner.
Move 37 remains a useful landmark because it makes that pattern concrete. A learned preference can guide exploration without dictating the final choice, and a method trained partly from human examples can exceed familiar human expectations in a bounded domain. The evidence comes from an identifiable system, game, date, opponent, and result. Preserving those details keeps the story connected to an experiment. Its lasting invitation is to investigate how learning and computation cooperate to discover effective actions, and to measure where that cooperation succeeds or fails when the rules become less accommodating.
Sources & further reading
Primary papers, project documentation, and research releases used in this article. Publication dates below refer to the sources themselves.
- Mastering the game of Go with deep neural networks and tree searchNature / Google DeepMindChecked
- AlphaGo’s ultimate challenge: a five-game match against the legendary Lee SedolGoogle / DeepMindPublished Checked
- AlphaGoGoogle DeepMindChecked
- AlphaGo Zero: Starting from scratchGoogle DeepMindPublished Checked
FOLLOW THE QUESTION
Study engineering foundations