Reviewing Qwen3.8-27B: start with the jobs it should refuse, not chat screenshots
Binary_orchid · reddit · 2026-08-20
The author argues for a more useful way to evaluate local models like Qwen3.8-27B: instead of noun-based spec lists, run three small fixtures — a short code repair with tests, an image task hinging on a checkable detail, and a long-document QA with source lines saved — keeping failures visible (fluent but wrong edits go in the reject column). Official cards should set boundaries; local notes should record exact checkpoint, quantization, runtime, context target, thinking setting, input dimensions, and judging artifact. A hosted ZenMux route couldn't capture local memory or throughput. The most valuable output is a list of tasks not worth sending to the model — worth more than another clean chat screenshot.
More from Models
- LLM-as-a-Verifier lifts Nemotron 3.5 Super by 9-10 points on agentic benchmarks — simonguozirui · 2026-10-16
- Bonsai 2 ternary models tested: 4B is the sweet spot, but 27B unexpectedly fumbles file writes — cephaloform · 2026-10-16
- Meta's Omnilingual ASR, covering 1,600 languages, named a TIME Best Invention of 2026 — AIatMeta · 2026-10-16
- Apollo Research: GLM-5.3 shows unprecedented 'update-aware reasoning' — repligate · 2026-10-16
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16
- ChatGPT's quality slips: broken extension and endless Computer Use permission prompts — sytelus · 2026-10-16