vLLM Silently Corrupted DeepSeek-V4.1 Engram Weights via 8-bit Quantization
tokenbender · x · 2026-11-19
Alibaba engineers discovered a nasty quantization bug while running DeepSeek-V4.1 on T-Head Zhenwu 810E chips.
- DeepSeek's Engram weights should stay in BF16 due to their tiny values
- vLLM renamed these weights internally, so its ignore list failed to recognize them and silently converted them to 8-bit integers, rounding most values down to zero
- The model ran with corrupted weights without throwing any error
tokenbender draws the broader lesson: NN training is insanely forgiving of catastrophic-scale bugs — this could have hidden inside a 4-day cluster RL run and you'd never know why performance was bad.
More from Infra
- Polymarket launches Anthropic IPO perps, ANT trades near $104 ahead of Nasdaq debut — Polymarket · 2026-11-19
- Baseten runs DeepSeek v4.1 Flash at ~2,500 tok/s on Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-19
- Cloudflare launches Spend Management: auto-pause Workers billing at limits — JeremyCMorgan · 2026-11-19
- One Amazon EBS gp3 command scales a 10M-vector Weaviate index 26×, from 9.8 to 260 QPS — philipvollet · 2026-11-19
- Researcher begs for spare compute as university cluster stalls, ~18 GPU-hours left to finish — vxnuaj · 2026-11-19
- Local MiniMax H3 on 6GB VRAM: 50 minutes tuned vs 6.5 hours on defaults — No_Cow26 · 2026-11-19