How MENACE Taught AI Principles Without a Digital Computer

In 1961, British artificial intelligence pioneer Donald Michie lacked access to a digital computer, so he built a learning machine out of 304 matchboxes and colored beads. Known as MENACE (Matchbox Educable Noughts and Crosses Engine), this mechanical device learned to master the game of tic-tac-toe through trial and error. This article explores how Michie’s simple contraption successfully demonstrated the foundational principles of reinforcement learning—states, actions, rewards, and punishments—decades before modern computing made advanced machine learning commonplace.

To understand MENACE, one must first understand how it mapped the game of tic-tac-toe. Michie recognized that accounting for board rotations and reflections, there are 304 unique board layouts that the first player might face during a game. He used one matchbox to represent each of these 304 possible states. On the front of each matchbox, he glued a drawing of the specific board layout it represented.

Inside each matchbox, Michie placed colored glass beads. Each color corresponded to a specific empty square on the tic-tac-toe grid. Therefore, the beads represented the possible legal actions the machine could take from that specific board state. If a board state had four empty squares, the matchbox contained beads of four different colors.

Playing a game against MENACE required a human operator to act as the mechanical processor. MENACE always made the first move. The operator would find the matchbox showing the empty board, shake it, and randomly draw one bead. The color of the bead dictated where MENACE placed its "X" on the board. The matchbox was then left open on the table with the drawn bead sitting beside it. After the human opponent made their move, the operator would find the matchbox corresponding to the new board state, draw another bead, and repeat the process until the game ended.

The brilliance of MENACE lay in what happened after the game concluded. The machine utilized a reward and punishment system that mirrors modern reinforcement learning algorithms. Because the used matchboxes and the chosen beads were set aside during the game, the operator had a complete record of the exact decisions MENACE made along the path to victory, defeat, or a draw.

If MENACE won the game, it received positive reinforcement. The operator returned the drawn beads to their respective matchboxes and added three extra beads of the same color to each box used in that game. This increased the statistical probability that MENACE would choose those same winning moves if it encountered those board states in the future.

If MENACE lost the game, it received negative reinforcement. The operator simply threw away the beads that were drawn during the match. By removing those beads from the matchboxes, MENACE became less likely to make those specific mistakes again. If a matchbox ever ran out of beads entirely, that specific path was permanently abandoned.

In the event of a draw—which is the optimal outcome between two perfect tic-tac-toe players—MENACE received a smaller reward. The operator returned the original bead to each used matchbox and added one extra bead of the same color. This encouraged the machine to favor draws over losses while still preferring outright wins.

Over the course of a two-day tournament against human players, MENACE initially lost frequently because its moves were entirely random. However, as bad beads were removed and good beads multiplied, the probabilities inside the matchboxes shifted. Within 20 games, MENACE was consistently beating or drawing against its human opponents.

Through this analog process, Michie physically manifested the core components of reinforcement learning. In modern artificial intelligence, an "agent" (MENACE) interacts with an "environment" (the game board). The agent observes a "state" (the matchbox), takes an "action" (drawing a bead) based on a "policy" (the ratio of bead colors), and receives a "reward" or "penalty" (adding or removing beads) that updates its policy for future iterations. MENACE proved that machine learning is fundamentally a mathematical process of adjusting probabilities based on past outcomes, requiring no silicon microchips to function.