Qwen3.8-27B Local Test: Fact-Extraction F1 Ties Qwen3.6, Decode Speed Drops 16%
KitchenAmoeba4438 · reddit · 2026-08-15
In a local fact-extraction head-to-head, Qwen3.8-27B scored 0.7030 F1 vs Qwen3.6-27B's 0.7177, a statistically insignificant difference (CI includes 0), so it's a tie. Decode throughput fell from 85.6 to 72.1 tokens/s (16%), but Qwen3.8 produced shorter answers, lowering end-to-end latency. The author notes Qwen3.8 shows massive gains on benchmarks it was trained on, but not elsewhere, questioning benchmark utility.
More from Models
- ThursdAI live from Manhattan: Muse Wonder, Muse Spark Open and Gemini 4 Argon — altryne · 2026-10-16
- Claude now leads 26% of Anthropic's model R&D, up from under 1% in February — altryne · 2026-10-16
- 'Small-Claude-shame': telling Opus 4.7 it's OK to know who you may ease it — repligate · 2026-10-16
- Bridgewater fine-tunes Qwen3-235B with Tinker, beats GPT-5.5 at 1/14th the cost — simonguozirui · 2026-10-16
- ChatGPT voice selection breaks after GPT-Live-1.1 rollout on Android — shaunralston · 2026-10-16
- Musk teases two-way Grok x Optimus integration: Grok in the bot, bot in Grok — beffjezos · 2026-10-16