LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Time
Yuxuan Zhang, Haozhong Xiong, Yubo Huang, Jiayi Song, Jinpeng Yu, Haofan Wang, Jiaming Liu, Ruihua Huang, Liwei Wang
cs.CV
2026-08-12
LiveAnimate reworks a 14B video diffusion model into a streaming generator, hitting 19.6 FPS on two H100s and holding identity steady over 3 minutes, where peer 14B systems need hours offline.
Pose-driven human animation generates a video of a target person from one reference image plus a driving pose stream. Live streaming, virtual avatars, and telepresence demand real-time response: every incoming pose must produce a matching frame without delay. Existing diffusion-based systems take minutes to hours per clip, which rules out interaction.
They also drift. Past a few seconds of generation, identity and appearance start to degrade. Real-time and minute-scale stability have never been achieved together in a full-body animation system at the billion-parameter scale.
LiveAnimate rebuilds Wan2.2-Animate-14B. The core move is turning a bidirectional diffusion model into a block-causal autoregressive generator, with a two-stage training pipeline plus a bounded cache for stability.
Stage 1, Reference-Anchored Teacher-Forcing Adaptation: the video is cut into temporal blocks, and each block is denoised while attending to clean ground-truth history rather than the student's own error-prone output. A global Ref Sink stores the reference image's key-value states, visible to all later blocks. This teaches the model to generate block by block.
Stage 2, Block-wise Self-Forcing Distillation: cuts per-block sampling from 50 steps to 3. The student first generates a full trajectory without gradients, then replays one block at a time with gradients on just that block. Trained with LoRA rank 128 on a single 8x80GB node.
PR-Sink (Pose-Retrieval Sink Attention) is what keeps long video stable. It is a bounded KV cache with three parts:
Because the cache is bounded, memory and per-block latency stay constant regardless of stream length.
On two H100s at 480x480: 19.63 FPS, about 611 ms per 12-frame block.
On a three-minute benchmark (25 FPS), from the first 30 seconds to the final minute:
| Segment | IQA | DINO-S | FID |
| 0-30s | 4.047 | 0.833 | 99.38 |
| 120-180s | 4.026 | 0.818 | 100.90 |
Nearly flat. By contrast, One-to-All's DINO-S falls from 0.769 to 0.328 and IQA from 3.402 to 1.786, with FID climbing from 146.29, its identity effectively destroyed. Peer 14B systems need several hours offline for the same three minutes (One-to-All about 2 hours); LiveAnimate streams in real time.
Scaling: 1 GPU 12.41 FPS, 2 GPUs 19.63, 4 GPUs 22.13.
This is the first full-body animation system to hit both real-time and stable long-form generation at billion-parameter scale. Prior real-time work used lightweight GANs and sacrificed quality; prior 14B diffusion work was offline-only. 19.6 FPS clears the interactivity bar, though only at 480p. That is a directly usable engineering target for live digital humans and virtual avatars.
The authors state the limits plainly: resolution capped at 480x480; three denoising steps caps visual fidelity; no multi-person scenes; no large camera motion.
One design choice stands out: the memory bank is written only during the first 20 blocks (about 8 seconds), so pose coverage is frozen after that. A genuinely novel pose appearing later has no matching appearance context to retrieve. The paper validates stability only out to three minutes, so whether it holds at the ten-minute scale is unproven.