“I felt like I was playing against a god of Go.” — Lee Sedol, world champion, 2016

In 1992, a researcher at IBM gave a computer program no strategy, no expertise and no knowledge of backgammon beyond the rules, then left it to play itself. One and a half million games later, it was beating the best players in the world and finding moves nobody had ever seen. Gerald Tesauro hadn’t taught it anything. He had just given it a way to keep score.

“Living organisms are consummate problem solvers. They exhibit a versatility that puts the best computer programs to shame.” — John Holland

In the mid-1950s, John Holland was first introduced to the mathematics of natural selection. Holland was a graduate student in mathematics at the University of Michigan who already knew how to program a computer, a rare combination at the time. He saw what nobody had seen. Evolution was a search algorithm that could be programmed.

What Holland saw was intelligence that searched rather than reasoned. At the time, you wrote programs by telling the machine what to do, step by step. Holland proposed the opposite: specify a test for what a good answer looks like, then let the program evolve toward it.

“The world is its own best model. It is always exactly up to date. It always contains every detail there is to be known.” — Rodney Brooks

In 1988, NASA held a workshop on how to explore other planets. The expected answer was one large rover, carefully engineered, remotely controlled from Earth. Rodney Brooks proposed the opposite: swarms of cheap autonomous robots, mass-produced, expendable, no human in the loop. To prove the idea could work, he built a walking robot in twelve weeks with two people and model airplane servos for legs. It now lives in the Smithsonian National Air and Space Museum.

“Probability theory is nothing but common sense reduced to calculation.” — Pierre-Simon Laplace, 1814

“When you have uncertainty, and you always have uncertainty,” Judea Pearl said, “rules aren’t enough.” Pearl had arrived at AI by accident. Semiconductors wiped out his job at a memory research lab in 1969, he called a friend at UCLA, and took whatever position was available.

He spent years reading the field’s approaches to uncertain reasoning — fuzzy logic, belief functions, certainty factors — and became convinced it was avoiding something it already had the mathematics for. Bayes’ theorem had been available since 1763. It told you exactly how to update a belief when evidence arrived. Why was nobody using it?

Here is the opening from a session called “The Dark Ages of AI,” at the world’s leading AI conference.

“In spite of all the commercial hustle and bustle around AI these days, there’s a mood that I’m sure many of you are familiar with of deep unease among AI researchers who have been around more than the last four years or so. This unease is due to the worry that perhaps expectations about AI are too high, and that this will eventually result in disaster.”

“Knowledge is the only instrument of production that is not subject to diminishing returns.” — J.M. Clark, 1923

Your best salesperson just resigned. Twenty years of knowing which people matter, who to speak to when, and how decisions actually get made. None of that knowledge is in the CRM, so when they leave, it goes with them.

Every organisation carries knowledge it can’t locate. It lives in the people who’ve been there longest, embedded in the culture and inherited processes. It isn’t written down because no one can tell you exactly what it is.

In 1965, Edward Feigenbaum, a computer scientist at Stanford, thought intelligence required knowledge. Within a decade, leading researchers were publishing findings using a system he designed, without thinking twice about the AI behind it.

“That moment was pretty symbolic to the world of AI because three fundamental elements of modern AI converged for the first time.” — Fei-Fei Li, 2024

In 2012, a graduate student trained a neural network in his bedroom on two gaming GPUs. It beat every major AI lab in the world.

“Nothing is more practical than a good theory.” — Vladimir Vapnik

Vladimir Vapnik arrived at Bell Labs from Moscow in the early 1990s already in his 50s. He brought three decades of statistical learning theory the Western world had never seen. From 1961 to 1990, he had worked on one question. Under what conditions can you guarantee a learning algorithm generalises from training data? Mathematics that the Cold War had kept invisible.

“Artificial intelligence must be based on real human intelligence, which consists largely of applying old situations—and our narratives of them—to new situations.” — Roger Schank

Intelligence requires memory. You cannot expect machines to learn without remembering what worked. Everyone building enterprise AI is grappling with the same problem. How do we give AI systems persistent memory without the bloat problem?

“When one encounters a new situation one selects from memory a structure called a Frame. This is a remembered framework to be adapted to fit reality by changing details as necessary.” - Marvin Minsky, 1974

Building enterprise AI means teaching an LLM the messy reality of your business. You need to explain standard contracts but also the edge cases. You need to describe typical customers and the exceptions that break the pattern. You need to capture default processes and how they vary by region. In 1974, Marvin Minsky wrote an essay about exactly this problem. His ideas shaped knowledge representation and drove the expert systems boom of the 1980s.

“Physical symbol systems are capable of intelligent action, and search is the essence of heuristic problem solving.” — Allen Newell & Herbert Simon, 1976

Problem solving involves considering different options, breaking things down, searching for a solution. Playing chess you explore options, consider possible moves, what your opponent may do in reaction, weighing up options along the way. It quickly becomes apparent that you can’t look at all options, so you focus attention and search intelligently. You rely on patterns and tactics picked up along the way.

Alongside knowledge representation, replicating intelligent search became the focus of early AI efforts. When we left Simon and Newell after Logic Theorist, they had recognized the core problem: you cannot search everything, so you must search intelligently. Inspired by George Polya’s work on problem solving, they built the General Problem Solver, demonstrating two important innovations.

“The central problem of artificial intelligence involves how to express the knowledge about the world that is necessary for intelligent behavior.” — John McCarthy

Arguably no one has had such a long lasting impact on AI as John McCarthy. He shaped the field in ways few others did and his contributions cast a long shadow for decades. He gave the field its name, posed central questions and introduced the programming language that made exploring them possible. His key insight was to consider how knowledge should be represented.

“Give me another six months and I’ll prove to you that it works.” — Geoffrey Hinton

As we saw in the Perceptron post, by the end of 1960s, funding for neural network research dried up and most researchers moved on to other promising approaches. Most, but not all. A small group of researchers believed.

“The first machine which is capable of having an original idea.” — Frank Rosenblatt, 1958

The history of AI is a history of ideas and failure. This is true from the very beginning. Where Simon and Newell demonstrated intelligence by solving problems, others believed learning was the key. An intelligent machine should adapt from experience, improve with practice, and learn from mistakes.

“Machines will be capable, within twenty years, of doing any work a man can do.” — Herbert Simon, 1965

January 1956. Herbert Simon’s living room in Pittsburgh. His wife, three children, and several graduate students stand in a circle, each holding a 3x5 index card carrying instructions. Each person is a component of a program that doesn’t yet exist on any machine. Simon gives a signal. They begin passing messages, following the rules on their cards, executing the program by hand. They are humans simulating a computer designed to simulate human thinking. Simon would later recall, “Here was nature imitating art imitating nature.” If the program they are running works, it will prove that machines can think.

The history of AI is a history of ideas and failure.

Ideas come from anywhere intelligence is evident. The field has borrowed from logic, neuroscience, evolutionary biology, cognitive psychology, even the behaviour of ants. Problem solving is how we recognise intelligence. So these ideas get tested by building systems to tackle problems that require intelligence. If a system handles a hard problem, it demonstrates something that looks like thinking. But once a system becomes useful, usefulness is what matters. Whether we still call it intelligent becomes irrelevant.