Yandex Research: KV-cache as a runtime for concurrent LLM interaction without retraining
RichmanRonald · x · 2026-09-17
Yandex Research published a blog exploring how sharing and scheduling KV-cache state lets pretrained LLMs observe, reason, and act concurrently without additional training.
- Problem: today's models process sequentially — they stop observing while reasoning and must restart to revise trajectories, which breaks down in games, robots, live video, and other evolving environments
- Existing approaches: Wan-Streamer and Thinking Machines Lab's Interaction Models agree sequential tool calls and turn-based streams are insufficient, but solve it by changing the model (pre/post-training, block-causal attention, streaming encoders, policy heads)
- Their take: most of this interactivity can be achieved at inference time — treating KV-cache state as a shared, schedulable runtime so observation and reasoning progress concurrently, with weights untouched
A reframing of interactive agent architecture from a training problem to an inference-time systems problem.
More from coding & agent
- Typing 'c' to approve: the inside jokes of AI coding agent prompts — burhop · 2026-09-17
- Dev lead running 50 deploys a day shares how to write and run tests in the vibe coding era — dotey · 2026-09-17
- YC built AI versions of its partners on GLM-5.2, cutting latency 31% vs OpenAI — ycombinator · 2026-09-17
- Shape Claude's Tools the Way You Want Instead of Hiding Behind Indirection — trq212 · 2026-09-17
- Out of Codex quota? Signing in with a new Plus account preserves all your chats — TheMoonMidas · 2026-09-17
- Vercel's fx may switch safety reviewer to Jev: 5-18x faster than GPT Luna — thesaraharminta · 2026-09-17