Sciforium hits 1,129 output tokens/sec P90 on DeepSeek-V4.1-Flash over 4 AMD MI355X GPUs
rohanpaul_ai · x · 2026-11-19
Sciforium reports DeepSeek-V4.1-Flash reaching 1,129 output tokens/sec at P90 on four AMD Instinct MI355X GPUs while keeping DeepSeek's original weights — about 10,000 tokens in roughly nine seconds.
Key techniques:
- A Traffic-tuned Drafter: speculative decoding drafters fine-tuned on real customer traffic, raising token acceptance and allowing draft length up to 7 tokens;
- Kernel work fusing small ops (mHC, projections) and fusing all-reduce with compute.
The team stresses this is production traffic, not a toy benchmark, answering doubts about AMD GPUs for inference.
More from Infra
- DNeo: Better Target Feature Encoding Speeds Up Speculative-Diffusion LLM Inference — cerebras · 2026-11-19
- vLLM v0.34.0 Lands: 1,061 Commits, Up to 1.95x Faster Decode on GB200 — vllm_project · 2026-11-19
- Cascade MoE on Bittensor beats Salesforce's Moirai2 on GIFT-Eval 19 days after launch — const_reborn · 2026-11-19
- Running untrusted agents on GKE: sandboxing, cold starts, and state — rseroter · 2026-11-19
- OpenAI models find up to 14% PPA gains on AMD GPU IP, improve custom Jalapeño chip — OpenAI · 2026-11-19
- A Local Model for Every Device: 6 Free Offline Models from Phones to Servers — alex_verem · 2026-11-19