Richard Sutton wrote The Bitter Lesson: the history of AI research shows that general methods leveraging search and learning beat specialized methods every time. Every time a researcher hand-codes knowledge into a system, it works for a while. Then someone throws more compute at a dumber method and wins. Forty years of this. Chess, Go, vision, language. The lesson is always the same.
Search is sensing. Casting wide, trying everything, brute force over the space. You don’t know what you’re looking for. You enumerate. You sample. You feel around in the dark until something works. This is expensive. It scales with the size of the space. But it finds things that no one predicted, because it doesn’t need a prediction — it just needs enough time.
Learning is intuition. Compressing what search found into a pattern that predicts without searching. You take a million examples and extract a rule. The rule is smaller than the examples. The rule is wrong at the edges. But the rule lets you skip the search next time, which means you can act faster than the space is large. This is what a neural network does. This is what a person does when they develop a gut feeling.
The duality between them is broken in a specific way, and the way it’s broken is the interesting part.
Intuitions are formed out of senses. They’re maps between sense impressions — take in a field of raw data, compress it, output a prediction. Given an intuition, you can figure out how it would feel by plugging it back in. You run the model forward and it produces something sensory: a next token, an image, a motor command. The intuition reconstructs the sense impression.
But the reverse doesn’t work. Given a sense impression, you can’t uniquely recover the intuition that generated it. Many different models can produce the same output on the same input. The mapping from intuition to sense is well-defined. The mapping from sense to intuition is not. Intuition is built from sensing, but sensing doesn’t determine intuition.
In category theory, this is the distinction between an arrow and an object. An arrow (a morphism) between two objects carries structure — it tells you how to get from one to the other. Given the arrow, you can recover the objects at either end. But given an object, you can’t recover the arrows that land on it. There are too many. The arrow reconstructs the objects. The objects don’t determine the arrow.
Search gives you the objects — the raw sense data, the concrete examples, the brute-force enumeration. Learning gives you the arrows — the maps, the compressions, the patterns. The Bitter Lesson says: invest in the objects. Get more data, more compute, more search. The arrows will form downstream. You can’t shortcut to the arrows by hand-coding them, because the arrows you hand-code will be the wrong ones. You don’t know which compressions are correct until you’ve seen enough raw material to let the correct ones emerge.
This is the same reason that in the Objective Personality System, sensing and intuition form a duality rather than a spectrum. They’re not more or less of the same thing. They’re structurally different operations. Sensing accumulates. Intuition compresses. Accumulation is reversible — you can always go back and look at the data. Compression is lossy — once you’ve extracted the pattern, the data that didn’t fit the pattern is gone. You traded resolution for reach.
The basic function flow is wide in, narrow, wide out again. You sense broadly, you compress into an intuition, and then you act on the intuition — which produces new sensory data, which starts the loop over. This is also a training loop: forward pass (sense), backprop (compress), next forward pass (sense again with updated weights).
The question is which one to trust when they disagree. Sutton’s answer is clear: trust the search. Trust the sensing. The intuition is always downstream and always revisable. If your gut says one thing and the data says another, update the gut. The hand-coded intuitions of expert systems, of symbolic AI, of every clever shortcut that seemed like it should work — they all lost to methods that just looked at more data.
But you can’t search forever. At some point you have to compress and act. The art is knowing when to stop sensing and start committing to a pattern. Too early and you overfit to noise. Too late and you never act at all.
The narrowing is the hard part. It’s where you decide which intuition to keep and which data to discard. Everything rides on the quality of that compression.
From my notes.