Nvidia Researchers Call Huge Context Windows a Dead End, Propose Note-Based Models
The Decoder · rss · 2026-11-22
Nvidia researchers argue that language models often don't need their full context windows, which are slow, expensive and error-prone, and propose an alternative.
- In tests, periodic summaries plus targeted keyword search usually matched full-context results at a fraction of the cost.
- The team goes further, proposing a new class of models that work with short notes instead of long context.
The takeaway: instead of endlessly scaling context length, move memory and retrieval outside the model.
More from Models
- Claude predicts AI will crack math proofs, battery materials and abandoned drug candidates within 5 years — alex_verem · 2026-11-22
- Researcher: OpenAI can't stop distillation at medium reasoning effort — adam_karvonen · 2026-11-22
- OpenAI's own data suggests hidden CoT barely matters for distillation at medium reasoning effort — adam_karvonen · 2026-11-22
- 9B model hits 59 tok/s on a 16GB GPU, but tool calls lag at 58% BFCL — AIFlow_ML · 2026-11-22
- Anthropic's $1.5B book settlement unpaid; call grows to raise it to $20K per book — TuhinChakr · 2026-11-22
- Can 16-32GB Local Models Match Closed-Source SOTA by End of 2025? — AnimalPuzzleheaded71 · 2026-11-22