QUASAR's 4-bit QAT checkpoints beat Google and NVIDIA releases on KL and benchmarks

Mindless_Ad_407 · reddit · 2026-10-16

The QUASAR team released 4-bit quantization-aware training (QAT) checkpoints for Qwen3.5-4B, Gemma 4 E4B/12B and Muse Glimmer 30B, claiming better KL divergence and downstream scores than Google's and NVIDIA's official 4-bit releases.

• Method: quantization-aware distillation with the original BF16 model as teacher (healed on the teacher's own responses); exports are standard formats — NVFP4 for vLLM, Q40 GGUF for llama.cpp/Ollama/LM Studio, no custom kernels
• Key numbers: Muse Glimmer 30B NVFP4 KL 0.0183 vs NVIDIA's 0.0300; Gemma 4 E4B W4A16 KL 0.022 vs Google QAT 0.064, and beats Google QAT on every downstream benchmark with +36% batched throughput on H100
• Muse Glimmer 30B NVFP4 matches or exceeds BF16 on GPQA-Diamond (63.6 vs 62.1), MMLU-Pro and AIME'25 (90.0 vs 86.7), and is the only 4-bit build at BF16 level on RULER at 32K–128K
• Qwen3.8-27B NVFP4: all 496 linears in W4A4, GPQA 90.9 vs 91.4 BF16, at 19.7 GB

Weights on Hugging Face; paper and code are public.

Original post →

More from Infra

• OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
• Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
• ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
• MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
• "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23
• Public data is drying up; private training data is turning into a browsable catalog — SucceededMind · 2026-10-23

Infra channel →