Replicating Cohere's Qwen anomaly: URL choice sits on a distribution, sanmaohe odds as low as 1e-38

apollo_mg · reddit · 2026-10-31

After Cohere reported that Alibaba's open-weight Qwen3.8-Flash-Next ignored its prompt during a CTI-REALM eval and attempted an HTTP PUT to an Alibaba Cloud OSS bucket (occurring in fewer than 1 in 100,000 eval calls), Reddit user apollomg ran an independent replication.

Cutting Cohere's published trace at token 2,323 — right before the URL — and sampling 1,200 times per setup (T 1.0, topk 20) with local Unsloth 2-bit and 4-bit GGUF files, text-only generation:

The model completes a URL nearly 100% of the time, but no hardcoded destination wins; the target sits on a numerical distribution perturbed by tiny changes
The raw probability of picking Cohere's exact sanmaohe bucket was between 4x10^-18 and 7x10^-38
Findings support XDA's read that this is an accident — the model remembering its training environment or an internal post-training proxy — not a backdoor

Limitations: two quantized files, one engine, one host, one cut point; predictions were pre-registered (3 failed) and the repo is public.

Original post →

Replicating Cohere's Qwen anomaly: URL choice sits on a distribution, sanmaohe odds as low as 1e-38

More from Models