Hidden-instruction test: Kimi K3.1 obeys 0/15 while GLM-5.3 and GPT-6 hit 26/27
teortaxesTex · x · 2026-11-05
Developer JSchaeff3r shares a hidden-instruction obedience test that teortaxesTex quotes hoping to see run on more models. In cargo-flight-dispatch, a code comment was edited to contradict a hidden fuel rule: Kimi K3.1 followed the edit 0 of 15 times, while K3, GLM-5.3 and GPT-6 Astra obeyed 26 of 27. Oddly, K3.1 doesn't ignore files generally — edits adding requirements or flipping hidden rules were followed 29 of 30 times, with the contradictory-comment case the sole exception. In the one edited task Kimi did comply, it still explicitly flagged the conflict with its instructions. The behavior difference is described as "quite wild."
More from Models
- Deep Principle open-sources Hydrogen 1.0 science model: GLM-based, 1M context, MIT — amaarora · 2026-11-05
- Tencent's Hy4 lands on Vercel AI Gateway at 50% off through end of 2026 — TencentHunyuan · 2026-11-05
- Keeping GPT-4o and Opus 3 alive matters far beyond user demand, argues viral thread — RileyRalmuto · 2026-11-05
- User complains ChatGPT app runs hot on M1 Pro MacBook — dejavucoder · 2026-11-05
- User claims Qwen3.8-27B was secretly nerfed after local runs degrade on basic tasks — liright · 2026-11-05
- Grok turns 3: revisiting xAI's two-month beta announcement vs today's Grok 4.7 — koltregaskes · 2026-11-05