Paper: CKA-QAD Significantly Improves Reasoning and Coding in NVFP4 Quantized LLMs

Aaaaaaaaaeeeee · reddit · 2026-08-10

The arXiv paper Beyond Output Matching delves into the accuracy recovery problem of large language models under low-bit quantization (e.g., NVFP4).

• The Problem: Traditional Quantization-Aware Distillation (QAD) relies solely on KL-divergence to match output distributions, which can mask internal representational degradation. This causes severe drift in intermediate activation geometries, especially in RL-post-trained models, leading to bottlenecks in reasoning and coding tasks.
• The Solution: The authors propose CKA-QAD, a CKA (Centered Kernel Alignment)-guided representational alignment method. It preserves internal representational geometry during distillation by aligning layerwise Gram matrices via a lightweight regularizer.
• Results: Experiments on Nemotron 3 Nano and Qwen3-4B-Thinking-2507 show that CKA-QAD substantially improves representational alignment and boosts downstream reasoning and coding accuracy with modest training overhead.

Original post →

More from Infra

• MLCommons launches MLPerf Endpoints, a benchmark that measures inference endpoints the way buyers actually purchase them — TheKanter · 2026-10-16
• Neocloud M&A decoded: CoreWeave, Nebius, Crusoe, Nscale all bet margins vanish beyond GPU scarcity — AccBalanced · 2026-10-16
• AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
• Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
• WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
• Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option? — Dave86ch · 2026-10-16

Infra channel →