Unsloth quants top agentic benchmarks: 1-bit GGUF scores 68.1 on DeepSWE, 10% ahead
danielhanchen · x · 2026-11-03
Unsloth benchmarked DeepSeek-V4.1-Flash-1017 GGUFs at 512K context on agentic tasks: its quants rank highest overall, with 1-bit Unsloth GGUF hitting 68.1 on DeepSWE, 72.2 on Terminal-Bench and 52.7 on AutomationBench — over 10% ahead of other quants, staying close to full precision even at IQ1M. Quoted commentary argues long multi-turn agentic tasks are the best way to evaluate quantization, since errors accumulate over long sequences while saturated easy benchmarks show nothing.
More from Models
- Claude Opus 5.5 is the main model behind this comedy AI, creator reluctantly admits — ivan_bezdomny · 2026-11-03
- Mistral's ML4 report: KV-cache-saving attention tricks regressed SWE-bench agentic scores — _AndrewZhao · 2026-11-03
- WaPo Investigation: LLMs Got Voters' Congressional Districts Wrong 48% of the Time — Polymarket · 2026-11-03
- fchollet: LLM writing's worst habit is turning every sentence into a punchline — fchollet · 2026-11-03
- User on xAI's $1,200/yr Grok plan still hits 100% weekly usage cap — eter_inquirer · 2026-11-03
- DeepSeek's releases may be the year's most important: run a whole business on <$50/month — EXM7777 · 2026-11-03