Why counting breaks LLMs: the minimal attention program needs two indices
ctjlewis · x · 2026-09-10
The author argues counting is the simplest task requiring intermediate steps — the only way to count many things is one by one, just like guessing jelly beans in a jar. In the follow-up, they show a simple 1, 2, 3 pattern isn't sufficient for arbitrary-length sequences: counting the r's in "strawberry" requires a second index to enumerate the r's (letters counted to 9, r's counted to 3, tracked separately). This is the minimal attention program for the task, with complexity 2: it needs two simultaneous indices.
Related event: Why LLMs Fail at Counting Letters in 'Strawberry'(2 posts)→
More from Research
- Cognition's Devin agents factor RSA-260, set new record; RSA-1024 cost pegged at ~$30M — StefanoGogioso · 2026-09-10
- Researchers Introduce Programmable Cellular Automata: CA Rules as Readable Code — SimonGColton · 2026-09-10
- Harmonic's Aristotle: an AI agent that proves software correct with machine-checked proofs — satnam6502 · 2026-09-10
- ProteinBlender: Free Open-Source Blender Addon for Publication-Quality Protein and DNA Animation — DeryaTR_ · 2026-09-10
- GPT-6 'Astra' Does 34 Math Steps in Latent Space, 4x More Than Sol — MaartenBaert · 2026-09-10
- Judea Pearl resurfaces his obscure 1978 paper on model complexity and credibility — yudapearl · 2026-09-10