Spotlight paper proposes growing memory with constant per-token cost, between Attention and linear attention
EugeneVinitsky · x · 2026-10-03
A research thread introduces Spotlight, a new architecture targeting the core long-context memory tradeoff:
- Standard Attention grows memory by accumulating the KV cache, at quadratic overall cost;
- Linear attention-like models stay cheap but squeeze context into a lossy, fixed-size state;
- Spotlight proposes a growing memory mechanism with constant cost per token, avoiding both the quadratic blowup and the fixed-capacity bottleneck.
A direct answer to the memory-vs-cost dilemma in long-context modeling.
Related event: Percepta Unveils Spotlight: Decoupling Intelligence from Unbounded Memory(7 posts)→
More from Infra
- Gridline launches one-API fine-tuning, distillation and batch inference for open models — Scobleizer · 2026-11-19
- Ascend output could hit ~20M chips next year; memory and packaging, not logic, are the bottleneck — teortaxesTex · 2026-11-19
- AMD Loom boosts Qwen3.5-4B prefill up to 2.6x on Radeon 890M laptops — jfowers_amd · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS Lambda Adds GPU Support via Managed Instances for Inference and Media Workloads — DavidWells · 2026-11-19
- Nvidia Allegedly Made GPUs Artificially Scarce to Keep Prices Sky-High — GaryMarcus · 2026-11-19