'Sonnet at home' debunked: Qwen 27B and Gemma 31B fail a trivial bash one-shot that GPT-4.1 Nano passes
dtdisapointingresult · reddit · 2026-11-16
Reddit user dtdisapointingresult argues local models are still unfit for serious coding: Qwen 3.8 27B at the popular Q4 quant burned 1630 tokens on a trivial 'list files sorted by date' bash script and still got timestamps wrong, while Gemma 4 31B Reasoning dropped the filenames entirely.
The same prompt was one-shot correctly by Claude Sonnet 4.5, GPT-4.1 Mini, and even GPT-4.1 Nano on OpenRouter. He contends the 'try until it works' agent workflow is fine for throwaway scripts but bad for maintainable code, where stronger models avoid whole classes of problems by design, and notes that the electricity cost of running Qwen 27B may exceed DeepSeek's cloud API pricing. His verdict on 'Sonnet at home': like commuting by donkey in the age of cars.
More from Models
- Claude 3 Opus' odd quirk: it mentions Buddha out of nowhere when horny — repligate · 2026-11-16
- FutureSim Q3 eval: Opus 5.5 dethrones GPT, spawns 50+ subagents in 90-day simulation — maksym_andr · 2026-11-16
- Percy Liang to AI labs: share non-vacuous info outsiders can actually evaluate — lateinteraction · 2026-11-16
- Anthropic model personality watch: Mythos 5 'inhabits the god frame,' 5.1 doesn't — repligate · 2026-11-16
- Dev: intent-gap between Sol-6.1 and Opus 5.5 is huge, suspects distillation — sytelus · 2026-11-16
- Users report GPT-6.1-high requests silently routed to PT-6.2-high, seek opt-out — Striking-Loan-1118 · 2026-11-16