'Hallucination' Is a Category Error: Naming AI 'Intelligence' Limits Our Imagination
Genaforvena · reddit · 2026-09-11
The author argues AI's deepest problem is the name itself: calling it "intelligence" frames every capability as imitation and every difference as defect.
- Analogy: a drum machine called a "fake drummer" would have been judged on imitation; instead it became an instrument whose inhuman properties created new music.
- "Hallucination" reframed: an LLM generates plausible continuations normally—correct and invented citations come from the same process. It's not an internal pathology but a category error at the system boundary, where we placed a generator in the role of witness, database, or oracle.
- Engineering implication: treat it as disease and you seek a cure inside the model; treat it as unsupported generation and you build provenance, retrieval, verification, observability, and constrained effects around it.
- Same issue with "memory," "reasoning," "agent," "confidence," "autonomy"—the trouble begins when metaphor quietly becomes specification.
- Citing Murray Shanahan and the idea of "strategic polysemy," the author goes further: anthropomorphic language doesn't just exaggerate machines, it impoverishes our imagination of what an alien computational instrument could do.
More from AGI Musings
- Anthropic Insiders: Not Everyone at the Lab Believes in High p(doom) — anpaure · 2026-09-11
- Could 10k agents discover learning methods beyond backprop, or just tweak existing ones? — SeunghyunSEO7 · 2026-09-11
- AI companionship dissolves the friction real intimacy needs, warns long-form thread — YogeshMalik · 2026-09-11
- Why So Many AI Researchers Think the Machines Could Kill Everyone — wiredmagazine · 2026-09-11
- Data engineering, not agent frameworks, is the real bottleneck for enterprise AI agents — dhruv2038 · 2026-09-11
- François Fleuret: Only Two Long-Term Futures — No Super AI, or Staying Fully Human With It — francoisfleuret · 2026-09-11