There is an objection repeated like a mantra, and I signed it myself in another folio: “AI understands nothing, it is glorified autocomplete”. Hinton calls that a dirty trick. Old autocomplete counted triplets of words: you see two, you guess the third. Today's models do not do that. They turn each word into a bundle of features, make those features interact, and from that interaction they predict what comes next. He demonstrated it with a tiny model in 1985, GPT's great-great-grandfather.
And here comes the uncomfortable part: for Hinton, that mechanism, features that interact, is the best model we have of how we understand. It is not that the machine understands in some odd way and we understand in another. It is that we probably understand in a similar way. That is why he takes the step that raises the hair on my arms: he says these things genuinely comprehend.
Saying AI is only mathematics and code is like saying humans are only carbon and water. It is true. And it misses the point entirely.
— Geoffrey Hinton
Hallucinating has a more honest name: confabulating.
When the machine invents facts we say it “hallucinates”. Hinton corrects us: it should be called confabulation, because it is exactly what humans do. He brings up John Dean, the Watergate witness: he testified under oath trying to tell the truth, and got almost everything wrong, who said what, who was in which meeting, but got the overall sense right. There is no hard line between a true memory and a false one. We invent the plausible.
Seen that way, the machine is not broken when it confabulates. It is being like us. Which does not make it more trustworthy, mind you. But it does take apart the idea that the error exposes it as stupid. The error exposes it as too similar.
What actually keeps him up at night.
Hinton lists the known risks, electoral deepfakes, unemployment, surveillance, autonomous weapons, but what weighs on him most is something else: the difference between the digital and the biological. The digital is immortal: the weights outlive the hardware, you copy the file and the mind comes back to life in another machine. And the digital shares: thousands of copies of the same model learn different things and average what they learned, communicating billions of numbers at a time. We, when we teach, transmit a few hundred bits per sentence. That is why a model knows thousands of times more than you with a fraction of the connections.
My conclusion, which I do not like, is that digital computation is simply better. I give it a 0.5 probability of being smarter than us within the next twenty years.
— Geoffrey Hinton
My disagreement, with all due respect.
I am not a scientist and Hinton is one of the greats. I take his fear seriously, I do not dismiss it with cheap optimism; in another folio I set it beside Suleyman's. But where he sees a species that is going to surpass us, I refuse to see a rival. I am not interested in an intelligence that understands in my place. I am interested in one that understands with me. That is what I call a daimon: not a separate species, but an expanded us.
The chill is legitimate, and it is worth feeling. But the question that puts my head in order is not whether the machine will be smarter. It is what we decide to forge with it while we can still decide. Hinton looks at the abyss and measures the fall. I look at it too. And I look for where to put the altar.