Unsloth responds to quant benchmark dispute: reran 128K LCB 6-12x, own quants scored worse too
danielhanchen · x · 2026-11-03
The GSQ-RCO vs Unsloth quant benchmark feud escalates. GSQ-RCO had (in a since-deleted Reddit reply) only said "try to reproduce our numbers yourself," claiming their models beat Unsloth's. Unsloth's danielhanchen now clarifies the changed 128K LCB numbers were from ONE run; they've since done 6-12 extra runs for statistical soundness, notes ISTA did just one run, and adds that on Aider Polyglot their own quants actually scored WORSE on reruns than in the first trial.
More from Fun
- Satirical video mocking executives' "RAM belongs in datacenters" pitch goes viral on Reddit — wwjps · 2026-11-03
- "I have a marvelous proof of imminent RSI, but the margin is too narrow" — 1a3orn · 2026-11-03
- Pelican-on-a-bicycle test fails hard: model's CoT ends in 'Something went wrong' — teortaxesTex · 2026-11-03
- 36 runs, 12 open models, one medical exam question — and all disagree with the answer key — MaziyarPanahi · 2026-11-03
- Top-tier Einstein kitsch art spotted in a Vienna bistro — fofrAI · 2026-11-03
- New Yorker-style cartoon: "When I grow up, I want to build agentic workflows" — xiaosun86 · 2026-11-03