LLMs are 'bags of contextually activated circuits, heuristics and algorithms'
xuanalogue · x · 2026-09-27
In a thread on mechanistic metaphors for LLMs, xuanalogue proposes thinking of models as 'bags of contextually activated circuits, heuristics, and algorithms.' He argues that understanding them only at the training level (next-token prediction + RL) is useful but punts on learned internal structure—and that this metaphor explains both LLM competencies and their failures despite massive training.
Related event: LLMs as Bags of Contextually Activated Heuristics(3 posts)→
More from Research
- kalomaze proposes testing which nanogpt tricks survive causal NTP over DCT coefficients — kalomaze · 2026-09-27
- AI-designed drug turns back 6 aging clocks; semaglutide extends mouse lifespan 12% — rand_longevity · 2026-09-27
- Could perfect monosemanticity enable training coding agents without verifiers or RL? — menhguin · 2026-09-27
- Local agent eval: challenger model wrote tool calls as prose 5/80 times — a silent failure class — Grimmoner · 2026-09-27
- TalkPlayData-backed conversational music recsys challenge at RecSys 2026 draws 41 teams — keunwoochoi · 2026-09-27
- A better metaphor for LLMs: bags of contextually activated circuits and heuristics — xuanalogue · 2026-09-27