ReVolta 0.1.2 brings DFlash speculative decoding to 4x V100, hitting 201 tok/s on code
Major_Ingenuity_6364 · reddit · 2026-10-24
A Reddit user released ReVolta 0.1.2, a vLLM fork running DFlash2 speculative decoding on 4x V100 32GB PCIe cards (no NVLink), likely the first public DFlash setup on Volta architecture.
- The drafter (incoai's DFlash2 checkpoint) drafts 8-token blocks from the target's hidden states; target is Qwen3.8 Flash-Next (AWQ) with 512K context, FP8 KV with CPU offload, TP4 + DCP2
- Benchmarks (greedy, 512-token replies): 201 tok/s single-request English code and 195 tok/s reasoning, both beating MTP4's 163; Chinese is slower (85-101 vs 100-105) since the drafter was English-trained; 8 concurrent requests slightly lag MTP (331 vs 357)
- KV pool grows 10% to 1.27M tokens; MTP stays default with DFlash opt-in
- Other fixes: tool calling, Anthropic messages endpoint, and large-image token budget handling
More from Infra
- GPU Thermal Throttling Is Why Some of Your Training Runs Are Mysteriously Slower — dejanseo · 2026-10-24
- Figure now processes 81 minutes of video per second, built 'a startup within a startup' — coreylynch · 2026-10-24
- Tencent's Hy-Nomas video world model runs natively in ComfyUI on 12GB VRAM — LatentSpacer · 2026-10-24
- mini-AGI: continual-learning model trained on 8GB GPU for $56 beats GPT-2 Small on enwik9 — Another__one · 2026-10-24
- Flappy founders on CNBC: co-optimizing the AI stack unlocks far more chip value — DavidCahn6 · 2026-10-24
- Fireworks ships 7 new open models in two weeks, betting on specialized intelligence — sophiamyang · 2026-10-24