Kimi K3 Coding Tests Near the Frontier
PawelHuryn · x · 2026-07-17
The author compared Kimi K3 alongside Opus 4.8, GPT-5.6, and Grok 4.5 using the same 8-task coding benchmark.
Test Results
- Out of the 7 tasks K3 completed, it matched or exceeded the other three models in 6 of them.
- It caught 14 out of 21 planted bugs, beating Opus's 12, while the other models caught no more than 7.
- However, it also reported 2 false positives, making it the only model to do so.
Key Issues
- On launch night, 2 tasks returned 0 tokens directly—not wrong answers, but no output at all.
- After rerunning 13 hours later, 1 of those tasks achieved a perfect 14/14, indicating massive variability in model availability at different times.
- As of posting, OpenRouter is still rate-limiting, and Moonshot's native API is showing an engine overloaded warning.
Conclusion
- This is a 2.8 trillion parameter model, claiming to be the largest open-weight model in history.
- But the author's verdict is blunt: The weights are open, but the capacity is not. Currently, average users can't run it, and even Moonshot itself lacks the infrastructure to support it stably.
Related event: Kimi K3 Coding Test Nears Frontier Models but Lacks Usability(3 posts)→
More from coding & agent
- diginomica: agentic automation hype is overdone, and 'The Hackening' of enterprise security has begun — jonerp · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Devin launches Security suite powered by Claude Mythos with $100M+ credits — DevinAI · 2026-10-26
- HackBench injects reward hacks into agent trajectories to stress-test monitors — DanielKhashabi · 2026-10-26
- Hugging Face ships hf CLI updates: deduped cache saves 19GB, faster buckets, retryable GPU jobs — yb2698 · 2026-10-26
- One prompt, full Blender animation: dev says he never touched Blender once — iamfakhrealam · 2026-10-26