Qwen 27B Hallucinates 90% of Domain-Specific Code, Developer Reports
JustinPooDough · reddit · 2026-09-15
A developer running Qwen 27B locally at Q8/BF16 (50 tok/s) reports hallucinated classes and APIs in over 90% of answers on his enterprise Java domain, while DeepSeek and Microsoft Copilot deliver working code on the first try 99% of the time.
He suspects Qwen is optimized for agentic use rather than retaining domain knowledge in its weights, and asks for harness/system-prompt/retrieval strategies to curb hallucinations — a datapoint that general models still lag badly on niche vertical knowledge.
More from Models
- Resemble AI ships DETECT-World, a physics-based deepfake detector with 99.5% audio accuracy — AiBreakfast · 2026-09-15
- Grok 4.7 Misses Target Again; 2.5T-Parameter Grok 4.8 Finishes Training — eyishazyer · 2026-09-15
- Writers Report Gemini Flash 3.8 Suffers Severe Long-Context Rot Despite 1M Token Window — Quenty1 · 2026-09-15
- Ai2 Chair's 3 Predictions: Open-Source AI Goes from Ideology to Infrastructure in 18 Months — billhilf · 2026-09-15
- Surgical VLM Leaderboard: All Frontier Models Fall Far Short of Specialized Models — ddonoho · 2026-09-15
- Why Surgery Benchmarks Reward Fine-tuned Small Models While Math Benchmarks Don't — ddonoho · 2026-09-15