Proteus paper: incremental memory activation fixes linear attention's fixed-state saturation
behrouz_ali · x · 2026-09-15
A new paper thread introduces Proteus, addressing linear attention's core weakness at frontier scale: its fixed-size memory state saturates as context grows because all capacity is exposed from token 1, so early tokens waste capacity and later tokens inherit a full state. The proposed incremental memory activation schedules how much memory is live: early bottlenecks force summarization over memorization, then blocks unlock one at a time to admit new information. Proteus partitions memory into E blocks with gated reads/writes; the authors call it the simplest instantiation of the idea.
More from Research
- Alison Gopnik ties her 'LLMs as cultural technology' framework to recent AI math advances — AlisonGopnik · 2026-09-15
- C3 AI's DIA paper accepted at EMNLP 2026, tops all seven SQL benchmarks autonomously — C3_AI · 2026-09-15
- Reward AI debuts Omnibody Hand & OMI-1, pushing hardware-algorithm co-design for cross-embodiment dexterity — zipengfu · 2026-09-15
- OM-1 robot foundation model zero-shots across arms and humanoids, trained on human data only — zipengfu · 2026-09-15
- Burkov: DPO, IPO, KTO, SimPO, ORPO Unified as Three-Axis Choices in One Framework — burkov · 2026-09-15
- AgentSTAR tracks and reconstructs articulated objects from monocular video through occlusion — notmahi · 2026-09-15