Working memory seems to scale with model size, according to one researcher's observations
repligate · x · 2026-09-20
An interesting behavioral observation from repligate: a model's working memory capacity seems to scale with model size.
He finds Opus models have somewhat smaller working memory than he does and need to serialize more — the models would sometimes remark that he seems better able to "hold multiple things in mind at once." Newer models feel roughly comparable to him, talking like someone who natively operates with larger working bandwidth, though it's unclear how much was serialized in unseen reasoning.
More from Models
- Founder says he built non-autoregressive decision models a year before TypeSafe's Jev, but was ignored — Paimaamu · 2026-09-20
- Claude Opus 4 may be going dark: Vertex endpoints return 404/429, quota hike denied in a minute — repligate · 2026-09-20
- ChatGPT update rumored to add Meetings, agent features and Code Review — arrakis_ai · 2026-09-20
- 100-run trolley problem test: Jev pulls the lever 99% of the time — FinanceYF5 · 2026-09-20
- Parallel-sampling decision model Jev routes in ~1s vs 4-14s for regular LLMs — TigerOk4538 · 2026-09-20
- Burkov disputes Stanford AI Index: does the US really lead with 59 notable models? — burkov · 2026-09-20