Gears and Ghosts

There are now two kinds of software: gears and ghosts. The best systems will combine them both.

In the early days of generative AI—which was last year, by the way—you couldn’t turn a corner without encountering a story about some boneheaded action taken by an AI, on behalf of some bewildered (and / or negligent) company. Giving away plane tickets or cars, making up words doctors didn’t say, suggesting the sale of pre-nibbled cheese. These are potent cautionary tales; saving a little money on your call center won’t do you much good if you turn around and recommend chlorine gas as a refreshing mocktail.

The natural reaction is that this is a “bug” that needs fixing. Can’t we just teach these models to tell the difference between truth and lies?

This is the wrong framing. I mean, it’s not wrong to want to make them more truthful, but you won’t succeed in any real way, because their nondeterminism isn’t a bug; it’s so inherent to the technology, it’s practically a feature. LLMs are outlandishly complex artifacts, and there’s no future world where you just “iron out the kinks”. They are guessing machines, and if you put them into situations where they can’t guess correctly, they will still guess.

Reports like this have become more rare in 2026—not because models no longer make mistakes, but because companies got wiser and put in guardrails (though, at least as of this writing, you can still use the Chipotle bot for free AI inference tokens).

OK, fine—if that’s not the right framing, then what is?

A Tale Of Two Softwares

Consider: there are now two fundamentally different kinds of calculation, and they work by radically different rules.

Traditional software performs calculation like a gear: deterministic, with teeth that always mesh in the exact same way. It’s so precise, in fact, that we can build towering stacks of software (like, say, the Linux operating system, or the internet) that actually (largely) work.¹ ¹

The other kind—generative AI—is not a gear. It is a ghost: a force that’s powerful, responsive, and reminiscent of human form, while still being obviously (and eerily) non-human. The words inside an LLM probably emerged from a living human’s mind once², and the technology of generative AI has given us the ability to summon them back from the void, as conversation partners and problem-solvers.

This sounds ridiculous, but LLMs are ridiculous. An AI foundation model is a sprawling, cavernous universe. It’s not huge in absolute physical terms—the whole thing would fit on a $100 hard drive from Best Buy—but inside, there’s a compressed, blurry representation of a vast slice of shared human knowledge. When you converse with it, you’re forging a single path through its infinite, labyrinthine web of concepts and impressions. The results are ghostly whispers, emerging from a process that’s not consciously experienced by a living being, but preserves the shape and the meaning of thought.

And which—by the way—is incredibly effective at a wide range of cognitive tasks (when used correctly). LLMs can write code, summarize information, plan and debate and digest; and they are outlandishly fast compared to humans. They’re rapidly reformatting whole fields of human endeavor.

The true gears are still there, of course, in the background, doing gear stuff—trillions of lines of code, running continuously and flawlessly, embedded in every process, tool, and device on earth. The gears go on gearing, unaware that ghosts have arrived at the party and are vying for a share of the computational pie.

Gears Are Terrible Ghosts

For as precise and faithful as gears are, they’re also crap at doing many important things because they are narrow and rigid. Their exacting logic makes them unsuitable for the fuzzy, nebulous work our brains do all day long. That shape in the distance—is that a bear? That email subject line—is that going to upset my boss?

Even tasks that seem straightforward—”hey, could you merge these two customer spreadsheets?”—are a pain in the ass to do with gears. If you try to solve problems like this with traditional code, you end up in a series of increasingly desperate heuristics and brittle special cases without end.

But an AI just knows how to do this stuff. It knows that names and titles aren’t just strings of characters, they’re meaningful and richly patterned. It knows Bob and Robert are the same name. It even knows that Margaret and Peg are the same name. It deals in probabilities as a native language, not a special case.

Before AI, people tried SO HARD to make gears do ghost things. The Cyc project spent decades manually encoding common sense so that programs could use it. Machine translation limped along as a bunch of massive lookup tables with rules bolted on, capable of translating “Where is the bathroom?”, but giving you word salad if you asked for a joke or a poem. Try writing code that can read a document and give you a summary; there’s no algorithm for “what matters” or “how similar is this thought to that one”. Gears can count words; they can find keywords, extract sentences, store data.

But working at the concept level? Understanding what something is about? That’s a ghost thing.

Ghosts Aren’t Very Good Gears, Either

The same disparity goes in the other direction, too. Ask an AI to multiply two large numbers (without using a calculator), and after a million FLOPs through the belly of a neural network, the guess you get out the other side will be wildly wrong. A calculator from 1972, on the other hand, will turn its little silicon gears and get the perfect answer, every time, in microseconds. LLMs have a sort of “fuzzy impression” of what multiplication is; the calculator actually performs the algorithm.

But it’s not just “bad at math”, like a 1960s Barbie doll. It’s bad at doing anything that needs to happen the same way every time. Try using it to file your taxes—run it 10 times, and you’ll get 8 “very similar” results. That’s just its nature—it’s specifically nondeterministic, and each token is generated with a little bit of chance thrown in, every time. When you ask the guessing machine something really clear and obvious (like “Who was the first president of the United States?”), sure, you might always get the same answer. But venture into any subject even slightly more complex or subjective and all bets are off.

This means that you absolutely cannot build a “towering stack” of software atop LLMs, because there’s just too much room for creativity and divergence. Gears don’t work if the teeth are spaced a little differently on each spin.

This is no knock on LLMs; it’s just how they work. They aren’t algorithms; they are apparitions.

Putting The Ghost In The Machine

The upshot of this new duality is that for any given goal, you have to choose: do you want to use a gear (fast, cheap, rigid) or a ghost (expensive, non-deterministic, flexible)?

People on the cutting edge—the self-described “AI natives”—are trying to do everything with AI. Don’t read your emails, they say; let a ghost read them and act on them. Don’t call your prospects; have your ghost pick up the phone. Don’t bother with databases and UI widgets and logic and all that nonsense; it’s ghosts all the way down.

Some of that might succeed, but I suspect most of it will fail for things that really matter, because it’s much harder to confidently reason about what a ghost is going to do. If you could be confident that they’ll always be “close enough”, that might be one thing; but in practice, they’re often egregiously, horribly wrong.

On the other hand, while the “no AI ever” crowd might have defensible moral high ground, there are just some kinds of cognitive work that ghosts do ... better. If you’ve ever written code by hand, watching a coding agent work makes that abundantly (painfully) clear. Same goes for any kind of detailed symbol manipulation work: searching through nebulous data, editing prose, etc. You can choose not to use LLMs for these things, but you can’t pretend your way is more effective (since, to quote Spurgeon, the LLM will have “gone ‘round the world while you’re still pulling your boots on”).

But this stark dualism obscures what I think might be a happier middle path: you can use them both, together. You can build software that is real, concrete, specific code ... right up to the point where it needs to look at some human language nonsense and make a decision about it, at which point you summon a little ghost, ask it to do that one thing for you, and then release it back into the bardo.

We can have our real software—our dependable precision gears, now crafted at 10 times the speed—but when the time comes to parse that messy web page, or figure out which option someone really wants—well, say the magic word and you’ve got a single-use ghost for the occasion.

1. When traditional software breaks—which it does all the time, of course—it’s because of wobbly programming, not because the rules of computation are themselves wobbly. It’s still made of gears, they’re just shoddy gears.

2. An ever-increasing amount of the training corpus is also AI generated at this point, of course, so that’s becoming less and less true; though you could also just say that those words took a more circuitous route from the human to you.