WAM-Diff2 Distills Autoregressive Autonomous Driving VLA into Diffusion for 15.1x Faster Decoding
机器之心 · wechat · 2026-08-19
Fudan University and Yinwang propose WAM-Diff2, a general pipeline that smoothly converts a mature multi-task autoregressive VLA into a discrete diffusion architecture, addressing two bottlenecks in autonomous driving VLAs: linearly growing serial decoding latency and exposure bias from teacher forcing. Built on a Qwen3-VL backbone:
- Block-Causal attention: tokens interact bidirectionally within blocks while causality holds across blocks; block size is a continuous dial between AR (B=1) and diffusion.
- Three-level hierarchical distillation: ① progressive block-size adaptation (1→4→8→16→32); ② paradigm-consistent block-wise distillation with symmetric JSD so the model learns to handle noisy intermediate states; ③ model-level distillation from an 8B diffusion teacher to a 2B student — teacher-student paradigm consistency yields 58.2% top-5 token overlap vs 51.2% with an AR teacher.
- System speedups: 2.8x from paradigm conversion alone (44.5→124.8 tokens/s), reaching 15.1x cumulative with custom FlashInfer kernels and CUDA Graphs, latency down to 1.5ms/token, peak throughput 673.4 tokens/s with under 0.5pp metric change.
Achieves 91.1 PDMS on NAVSIM, beating specialized planners like ReCogDrive (90.8); 49.55% closed-loop success on Bench2Drive; long-horizon waypoint L2 error reduced 5.8% versus the AR baseline, suppressing error accumulation over the prediction horizon. Limitations: token quantization error and a student capability ceiling set by the AR teacher.
More from Research
- Differentiable Voxelization Links Surface and Volume Representations — ssh4net · 2026-08-19
- PoLoRA Optimizer Accelerates LoRA Finetuning, Boosts Stability — deepcohen · 2026-08-19
- Classic Paper: The Surprising Creativity of Digital Evolution — Ghost_Pilot_MD · 2026-08-19
- HarmProfile Benchmark: Harmfulness and Diversity Rise with Model Capability — Zhouyuan Ma · 2026-08-19
- Why Removing the Vision Encoder Can Be Better — From an Infra Perspective — liuziwei7 · 2026-08-19
- Matmul Optimization Bottleneck: Data Movement, Not Multiplications — yaroslavvb · 2026-08-19