Yandex Makes Pretrained LLMs Interactive by Reorganizing KV Cache, No Retraining Needed
arpit_bhayani · x · 2026-09-07
Yandex Research turned the KV cache from a mere speed optimization into an agent runtime. By letting multiple readers consume the same shared memory in different orders — no weight changes — pretrained LLMs can observe, reason, and act concurrently. A demo agent played Doom in real time, processing a constant frame stream, despite never being trained on the game. The takeaway: interactivity can come from rearranging how a model's memory is organized and exposed, a far cheaper lever than retraining.
Related event: Yandex Turns KV Cache into an Agent Runtime for Real-Time LLM Interaction(3 posts)→
More from Infra
- vLLM brings speculative decoding to AMD MI300X and MI355X, verifying multiple candidate tokens to cut latency — petrusenko_max · 2026-09-07
- Samsung reportedly co-developing on-device chip with Arm, with OpenAI as end customer — morqon · 2026-09-07
- Hugging Face datasets gets major streaming shuffle speedup via optimized Arrow C++ — lhoestq · 2026-09-07
- Qdrant calls out Actian's embedded vector DB comparison for its own "not embedded" pick — qdrant_engine · 2026-09-07
- Greenland pays $124/month for 15 Mbps as Starlink offers 200 Mbps in Denmark for $57 — XFreeze · 2026-09-07
- Red Hat and Hugging Face host PyTorch systems night in Bengaluru with 170+ contributors — PyTorch · 2026-09-07