Samuel's Checkers Program
Arthur Samuel develops IBM's first checkers program, pioneering the concept of machine self-learning.
Overview
In 1952, IBM's Arthur Samuel developed the world's first self-learning checkers program. This program could not only play checkers but also continuously improve by analyzing games. Samuel introduced the term "Machine Learning" and designed key technical innovations: learning from endgame positions backwards and evaluation functions based on game tree search. By 1962, the program could defeat checkers champions. Samuel's work was a pioneering contribution to machine learning, demonstrating that computers could improve their performance through experience.
Study Notes
Background, mechanics, impact, and key takeaways in one read
Background & Context
In 1952, IBM's Arthur Samuel wrote a checkers program for the IBM 701, one of the company's first commercial computers. A radio engineer by training rather than a chess master, he chose checkers over chess because its simpler rules and smaller branching factor let him focus on a bolder question: could a machine improve itself? The prevailing approach was hand-coded rules plus search, but Samuel wanted to test whether a program given an evaluation function and trial-and-error mechanisms could get better without being taught move by move by a human. This motivation directly gave rise to the very term and paradigm of "machine learning." Demonstrated on television in 1956, the program caused a sensation and reportedly nudged IBM's stock upward.
Key Points
- 1Samuel completed the first checkers program on the IBM 701 in 1952 and added learning by 1955.
- 2Building on Shannon's minimax, it scored positions with a learned "scoring polynomial" rather than fixed rules.
- 3Rote learning stored each seen position with its backed-up minimax value, effectively deepening the search on later encounters.
- 4Generalization learning had two copies play each other and adjusted the weights by outcomes, anticipating later TD methods.
- 5It used early alpha-beta-like cutoffs to cope with the tiny memory of the IBM 701 during its search.
- 6In 1962 the program beat Robert Nealey, a Connecticut state checkers champion, in a public match.
How It Works
Samuel's program looked ahead a limited number of plies from the current position and scored terminal boards with a "scoring polynomial" (a linear function approximation). It followed Shannon's minimax: backing values up from scored leaves, each position receives the score of the best move assuming the opponent uses the same criterion—what Samuel called the backed-up score. Two learning mechanisms are the essence. First, rote learning stored each encountered position with its backed-up value; on reuse it cached prior searches, effectively deepening search, and a tiny discount per ply biased the program toward "win sooner, lose later." Second, generalization learning had two copies (Alpha, Beta) play many games and nudged feature weights so that successive position values stayed consistent—the prototype of later temporal-difference learning.
Impact & Legacy
Samuel's work was among the first successful self-learning programs, proving a machine could improve without every step being explicitly programmed. His "scoring function plus self-play weight tuning" anticipated modern reinforcement learning—specifically Sutton and Barto's temporal-difference learning and Tesauro's TD-Gammon. Its limits were that play never reached master level and the learning lacked terminal rewards, so the piece-advantage weight had to be fixed by hand to avoid useless identical valuations. It also paved the way for the very coinage of "machine learning."
Key Takeaways
- "Learning" can mean continuously improving a scoring function, not memorizing answers.
- Self-play lets a machine improve from win/loss feedback—the seed of reinforcement learning.
- Without a correct objective signal, learning can collapse to a trivial "learns nothing" solution.
Glossary
Storing seen positions with their minimax backed-up values, then reusing them—effectively deepening the search.
Samuel's linear weighted sum that scores board positions; its weights are tuned by self-play outcomes.
Learning by the difference between successive state values; Samuel's generalization learning already hinted at it.