Spotlight architecture adds growing memory to LLMs with constant-cost token generation
teortaxesTex · x · 2026-10-03
Christos Tzamos's team unveiled Spotlight, a new architecture claiming LLMs can gain knowledge and capabilities without retraining: models get growing memory while token generation stays constant-work regardless of stored information, marrying attention's capacity with the linear cost of fixed-state recurrent models.
Commenter teortaxesTex notes the constant factor looks large — each access touches 9 cells, each storing a full dk×dv matrix per head — though that may be inherent to the continual-learning design.
Related event: Percepta Unveils Spotlight: Decoupling Intelligence from Unbounded Memory(7 posts)→
More from Models
- Three Google flagship models in eight days fuel speculation about hidden RSI work — haider1 · 2026-11-19
- Mystery Model 'Gestalt' Tops New Frontier Design Benchmark by 214 Points — soleio · 2026-11-19
- mrfakename releases abliterated GLM-5.3-Flash and Qwen3.8-Flash-Next — mrfakename0 · 2026-11-19
- Physician: AI competition now measured in weeks, GPT-6 versions 7 days apart — ramos_casals · 2026-11-19
- micro1's flow qc model screens training data for benchmark leaks before release — Exp_Mark · 2026-11-19
- Leaker predicts new DeepSeek paper this week, scheduled for November 11 — teortaxesTex · 2026-11-19