Yandex Publishes 'The KV Cache as an Agent Runtime': Inference-Time Interactivity for LLMs
arpit_bhayani · x · 2026-09-07
Yandex Research published the full write-up behind the recent KV-cache buzz: 'The KV cache as an agent runtime.' Interactive models must take in new information mid-computation, revise trajectories, emit partial actions, and coordinate processes running at different rates — unavoidable for games, robots, live video, and OSes. While Wan-Streamer and Thinking Machines' Interaction Models solve this via retraining, new data formats, or streaming encoders, Yandex explores how much interactivity can be achieved purely at inference time by sharing and scheduling KV-cache state, letting pretrained LLMs observe, reason, and act concurrently without any training.
Related event: Yandex Turns KV Cache into an Agent Runtime for Real-Time LLM Interaction(3 posts)→
More from Infra
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Cloud release velocity scores: AWS 95, Azure 90, Google Cloud 82 — DavidLinthicum · 2026-10-27
- Selling a 5090 to scale local inference: one business weighs 5 hardware paths — Anarchaotic · 2026-10-27
- AWS tutorial: fleet-wide edge CV model lifecycle with SageMaker SDK v3, ~$0.07 per run — AWS ML Blog · 2026-10-27
- Qwen3.8-27B hits 40 tok/s on Jetson Thor with INT4 + MTP, accuracy beats FP16 — No-Dragonfly6246 · 2026-10-27
- Celesto launches CI runners nearly 90% cheaper than GitHub Actions, with agent sandboxes — aniketmaurya · 2026-10-27