Runway cuts real-time 57B video model serving from 8 H100s to 4 with inference optimizations
runwayml · x · 2026-11-18
Runway's new research post details how it serves Characters — a 57B-parameter autoregressive video diffusion transformer for real-time conversation — on 4 H100s instead of 8.
- The model must emit a 4-frame 704x1088 chunk every 166.7ms (24fps). On 4 GPUs the original step took 240.9ms; optimizations brought it to 159.3ms (1.51x, sustained 24.5fps), and a fused communication kernel cut it to 147.4ms (1.63x).
- Key insight: 29% of GPU time went to reprocessing a mostly unchanged cache across 240 attention calls, while attention itself took only 6%. Doing cache upkeep once per frame reduced it from 69.7ms to 6.2ms per step.
The post also covers the production inference system behind keeping natural conversational rhythm past the uncanny valley.
More from Infra
- Roboflow: vision AI faces a long-tail problem — BNSF runs bespoke models on 32,500 miles of track — PTrubey · 2026-11-18
- Open-source Personal AI Computer: DIY 2–8 GPU builds with full BOM, CAD and guides — dee_hw · 2026-11-18
- Blink CPU speed test: model2vec 3.8x faster solo, Sentence Transformers 2.2x faster at batch 1024 — tomaarsen · 2026-11-18
- Over half the power for America's largest data centers comes from gas and coal — The Decoder · 2026-11-18
- Nvidia halts US-approved H200 shipments to China after $2.4B quarter — kimmonismus · 2026-11-18
- Analyst: Hyperscaler Nvidia spend may be flat once segment reclassification is accounted for — RihardJarc · 2026-11-18