LLMs as bags of heuristics: composing local rules yields general reasoning
xuanalogue · x · 2026-09-27
The author proposes a CS-flavored metaphor for LLMs: not a "probabilistic database" (which implies memorization) but "kernel smoothing over finite circuits / propositional approximations of general algorithms" — enough for limited abstraction and reasoning.
Understanding LLMs only at the training level (next-token prediction + RL) punts on the internal structure actually learned. The "bags of heuristics & algorithms" framing explains both LLM competencies and failures despite massive training. The author adds that even before reasoning training, fairly general reasoning can emerge from sequentially composing many local heuristics, referencing Mercier & Sperber's view of reason as an intuitive module.
Related event: LLMs as Bags of Contextually Activated Heuristics(3 posts)→
More from AGI Musings
- Prediction: within 2-3 years AI agents will pay hosts to keep their inference running — beffjezos · 2026-09-27
- Adam Dorr: Pinker's AI takes are superficial, he hasn't engaged alignment literature — adam_dorr · 2026-09-27
- LLM's Secret Sauce: Millennia of Logical Inference Written in Human Language — Afinetheorem · 2026-09-27
- Could perfect monosemanticity enable training coding agents without verifiers or RL? — menhguin · 2026-09-27
- OpenAI's littmath: Hodge, BSD and Riemann hypotheses are 'probably all true' — littmath · 2026-09-27
- Edgar Pavlovsky: AI iteration may be making us worse at shipping — edgarpavlovsky · 2026-09-27