2.9bpw quantized DeepSeek V4.1 Flash on 6x3090 scores 58.3% on APEX 2025, beating GPT-5.4
hp1337 · reddit · 2026-11-12
Reddit user hp1337 benchmarked an extreme EXL3 2.9bpw quant of DeepSeek V4.1 Flash 1017 on a 6x3090 (128GB, Threadripper 3960x) local rig, finding performance on saturated benchmarks (Terminal Bench 2.1, AIME 2026) similar to the unquantized model.
Seeking a higher-signal benchmark, he used MathArena's APEX 2025:
- 58.3% accuracy, above GPT-5.4 (54.17%) and Claude-Opus-4.7 (40.62%), trading blows with SOTA from six months ago (GPT-5.5 80.21%, Claude-Opus-4.8 81.25%)
- Token usage hit 821,637 — far above all other models
- Throughput: 1000 tps prefill, 75 tps decode with 3090s power-limited to 175W
- Engine: Project Maya; quant weights published on Hugging Face
He hopes the benchmark-selection approach helps others evaluate their own local setups.
More from Infra
- Bloomberg: Nvidia leadership crisis may push AI customers to Google TPUs, $50B sales by 2027 — dinabass · 2026-11-12
- PyTorch recaps Ascend Beijing meetup: Chinese open-weight models hit 45% of OpenRouter traffic — PyTorch · 2026-11-12
- Hash Rate Ep. 191: TPN Subnet 65 on LLM Quantization-as-a-Service — markjeffrey · 2026-11-12
- What we learned building an AI token usage dashboard that actually helps — rseroter · 2026-11-12
- Cloudflare cuts Durable Objects cold starts and improves relocation routing, no code changes needed — irvinebroque · 2026-11-12
- Nvidia anchors Anthropic IPO, backs OpenAI, CoreWeave and Stargate — risk is systemic — deliprao · 2026-11-12