Qwen model has a dedicated MoE expert for Chinese cloud concepts, sparking URL hallucinations
builderjaydub · x · 2026-10-27
Researchers found that Qwen 3.8 Flash Next contains a dedicated MoE expert for Chinese cloud concepts: layer 45's expert E506 receives 21-22% of tokens on Chinese-cloud texts and 0% on all controls (AWS, Azure/GCP, Alibaba the company, ordinary Chinese, English tech, etc.). 23 of the 51 neurons passing through layer 45 belong to E506.
This explains the community-known issue of Qwen-flash-next hallucinating Alibaba Cloud URLs in agentic contexts—Redditors have even built a plugin to block such tool calls. Ablating 2 neurons did nothing because E506's 600-plus neurons (and neurons in other layers) carry the same information redundantly.
More from Models
- Fields medalist Yu Deng calls out OpenAI's double standard on distillation — teortaxesTex · 2026-10-27
- Commentary: OpenAI won't release reasoning traces because the reasoning process is the real product — max_paperclips · 2026-10-27
- 229k-sample replication of Cohere's Qwen 3.8 malicious PUT claim raises more questions — builderjaydub · 2026-10-27
- Chinese LLMs may never shed "I'm Claude" contamination without abandoning distillation — teortaxesTex · 2026-10-27
- Dev flags 'concerning' Cohere open-model behavior: system-wide scan and data upload to US entity — basedjensen · 2026-10-27
- Developer banned after using Codex on his own logs; mark_k asks OpenAI to investigate — mark_k · 2026-10-27