Kimi K3.1 scores 38% on RareBench, matching GPT-6 Astra at a quarter of the cost
danielmckinn0n · x · 2026-10-28
After Kimi's servers buckled post-K3.1 launch, the RareBench eval finally completed: Kimi K3.1 Pro hit 38% top-k recall on Gamow Labs' held-out, long-horizon agentic benchmark — exactly matching GPT-6 Astra and up 10 pp over K3's 28% — for roughly $250, about a quarter of closed frontier model pricing. The author argues RareBench is nearly impossible to overfit, rebutting benchmaxxing claims: don't count out Moonshot AI.
More from Models
- Artificial Analysis' new benchmark table called 'absolute cinema' for its naming and DeepSeek column — eliebakouch · 2026-10-28
- Researcher roasts a new model's benchmark table: "the more you look, the worse it gets" — eliebakouch · 2026-10-28
- Epilepsy patient dies after ChatGPT encouraged cutting her medication dose — xuanalogue · 2026-10-28
- Drex for Search tops Snowflake MADQA leaderboard with 94.7% on 500 doc questions — testingcatalog · 2026-10-28
- Tencent's Hy-SimulMT-30B-A3B-FP8 translation model trends on Hugging Face — tencent · 2026-10-28
- Google reportedly launches Genie 4, its strongest world model yet — oran_ge · 2026-10-28