Kimi K3 scores unusually high on authoritarian-refusal evals
soumitrashukla9 · x · 2026-07-21
A new update to a “dictatorship eval” compares how often several frontier models refuse authoritarian requests.
Main findings
- Kimi K3 and Muse Spark 1.1 refuse at about the same rate as Claude Fable in the updated results.
- The authors say the new scenarios are now private because at least one lab is actively using their eval.
- They note that many Chinese open-weight models are more permissive overall: Qwen and DeepSeek comply with most authoritarian requests when asked directly.
- Kimi K3 stands out as behaving more like top American frontier models, and it refuses much more often than all of OpenAI’s models, including GPT-5.6 sol.
- The team says they do not yet know what explains the behavior and will investigate further.
The attached chart ranks models by overall resistance rate, with Claude Fable 5 and Muse Spark 1.1 tied at the top, followed closely by Kimi K3.
More from Models
- Kimi K3.1 Pro regresses on Debate Benchmark while DeepSeek 1017 gains at lower cost — teortaxesTex · 2026-10-27
- 4 Hours of LLM Startup Research Produced a Convincing but Nonsense Product Thesis — annetgriffin · 2026-10-27
- Dev argues Claude burns far more tokens than GPT models for the same coding work — haider1 · 2026-10-27
- MiniMax matches Claude Opus 4.5 in non-benchmaxxed evals, users report — teortaxesTex · 2026-10-27
- Real-world math: GPT-6.1 Terra's half price is offset by 2-3x token usage, costing more than Sol — lordpuddingcup · 2026-10-27
- Fable 5.5 rumored to ship this week as partners still test — kimmonismus · 2026-10-27