LLMs are General Asynchronous Agents
George Yakushev, Denis Mazur, Vladimir Bartenev, Vyacheslav Zhdanovskiy, Timofey Byzov, Vladimir Kaurkin, Vadim Pastushenko
cs.LG, cs.CL
2026-09-28
AsyncLLM runs Qwen 3.x as overlapping KV/GDN coroutines with no fine-tuning. SoccerNet AUROC 0.677 vs Mage-VL 0.555; monitoring uses about half the forward passes.
LLM agents still run a turn-based loop: read, think, act or call a tool, wait. A voice assistant has to listen while it thinks and handle barge-in. A monitor has to watch logs as they arrive. A robot cannot freeze until a long chain of thought finishes. The world does not wait for the next user message.
Each of those settings has been treated as its own research program. Full-duplex speech models, streaming video stacks, actor-thinker VLAs, asynchronous tool calling: each trains its own weights and ships its own inference code. This paper asks whether asynchrony can be a general skill, the way tool use already is, using off-the-shelf models and no extra fine-tuning.
AsyncLLM is an asyncio programming model for LLM inference. A developer, or the model itself, writes several inference coroutines. Each coroutine writes into a CacheBlock, a slice of internal state for a contiguous span of tokens: attention KVs plus, for hybrid layers, the affine transition of a Gated Delta Network (GDN), a fixed-size linear-attention recurrent state.
A forward pass writes one block. It can attend to many blocks, in any order, through a cache view. Think of a shared whiteboard with separate columns: each coroutine writes in its own column and can read the others as they fill in. If those blocks were encoded sequentially, the view matches a normal KV cache. When coroutines update blocks while looking at each other, the state no longer equals any serial run; in the experiments the model still treats it as readable context. Coroutines do not resend tokens to each other. A newly written token is immediately visible in shared attention.
Three layer types need different stitching:
The engine sits on mini-SGLang. Prefill and decode from different coroutines share a batch. Long prefills are chunked so a fast reaction coroutine is not stuck behind background thinking.
The streaming-video agent is five concurrent pieces: a per-frame event probe, a background thinker, an output probe, a description writer, and a non-LLM compiler. The probe is the same base model with a prefilled prompt, one forward pass, no extra training. The game agent reuses that skeleton: the thinker plans a maneuver, a faster action coroutine turns it into per-frame buttons, with an early-exit probe.
Two sanity checks come first. MATH-500-Sharded splits 500 math problems into an incomplete prompt and a mid-reasoning clarification. ShardedVQA builds 513 image pairs: the agent starts from an edited wrong image and, after k decode steps, is handed the corrected one. Qwen 3.x hybrid models absorb both text and image corrections while they think. Visual accuracy falls faster as k grows, because those items need shorter reasoning and large k often arrives after an answer is already on the page.
Streaming video is compared with Mage-VL, a model trained for this job:
| Method | SoccerNet AUROC | SoccerNet TriggerAcc | ProactiveVideoQA PAUC | TriggerAcc |
| Qwen3.5-9B | 0.677 | 62.82 | 0.541 | 52.99 |
| Qwen3.8-27B | 0.608 | 54.88 | 0.504 | 52.45 |
| Qwen3.6-35B-A3B | 0.651 | 62.55 | 0.539 | 52.30 |
| Mage-VL | 0.555 | 52.79 | 0.428 | 43.27 |
The official SoccerNet protocol does not score caption quality. On SoccerNet TimVal, the 9B model scores 39.62 against Mage-VL's 27.87; on ProactiveVideoQA overall TimVal the gap is 17.19 vs 10.03. Larger Qwen models also write more accurate descriptions on ProactiveVideoQA. The paper treats that as a side effect of using a generalist MLLM.
On ViZDoom HealthGathering and DeadlyCorridor (frame skip 4, 100 episodes) with Qwen3.6-35B-A3B, a human-written AsyncLLM agent takes fewer forward passes from observation to action than a sequential agent, while keeping most of the reward that reasoning provides. A probe-only baseline is faster and scores lower. The paper does not tabulate per-point rewards from those plots. It is explicit that this is a capability check, not a state-of-the-art game agent. Letting the model write its own coroutines from the environment spec works on HealthGathering and is worse on DeadlyCorridor. That path is not reliable yet.
System monitoring uses 34 DevOps-Gym Monitoring tasks. Logs are chunked by line, capped at 1 second or 1024 characters. Every method gets an anti-distraction prompt that restricts attention to resource issues. For Qwen3.6-35B-A3B:
| Method | Accuracy | Forward passes | Events seen at answer |
| Sequential agent | 61.76% | 8452 | 100% |
| Early answer per chunk | 47.06% | 6641 | 58.71% |
| Skip-row filter | 26.47% | 3686 | 65.86% |
| AsyncLLM | 55.88% | 3837 | 55.49% |
The 9B copy of the same setup: AsyncLLM 41.18% vs 47.06% sequential, 2582 vs 8726 forward passes. Without the anti-distraction paragraph, 35B-A3B overthinks unrelated issues, falls behind the log stream, and drops below 10% accuracy. The monitoring agent keeps 2.06 active coroutines on average.
On one H200, synthetic decode throughput in the paper's fastest reported row goes from 106 tok/s with one coroutine to 552 tok/s with eight. On ProactiveVideoQA the stack runs at about 2.5-4.4× realtime for video-only streams and 1.4-1.8× with audio.
Concurrent agents do not require a new speech model or a streaming VLM for every setting. A team that can run Qwen 3.x and write asyncio can express "think while watching" or "summarize while monitoring" as ordinary coroutines. Yandex, Together AI, and HSE released a reference implementation.
This is transfer, not a new base model. Beating Mage-VL on video is partly a stronger generalist writing better descriptions. The scheduler is not separately shown to dominate specialist event detectors. The games are not SOTA. Monitoring trades about six accuracy points for roughly half the forward passes. The useful case is real concurrent I/O. Static benchmarks will not show much.
Self-rewriting coroutines are a hint, not a product. If later training treats asynchrony the way current models treat tool use, the API shape changes: ship an agent definition into a live environment instead of one request, one reply.
The paper already says self-written coroutines are unstable, the games only show that the loop runs, and a generalist does not always beat a specialist on that specialist's turf. The ethics note adds a sharper failure mode: a fast coroutine can emit a wrong or harmful action before a slower reasoning coroutine can stop it. Self-modifying agents needed a sandbox.
The experimental base is almost entirely Qwen 3.x. SoccerNet does not grade commentary quality, and the Mage-VL win has no ablation that splits probe quality from description quality. ShardedVQA's wrong images are synthetic; generative models helped create and clean them. Monitoring is 34 tasks and leans on that anti-distraction prompt. There is no full-duplex speech experiment and no real robot. The claim that mutually updating cache views remain legible is supported by these tasks, not by a harder correctness argument.