Kimi K3 Coding Tests Near the Frontier
PawelHuryn · x · 2026-07-17
The author compared Kimi K3 alongside Opus 4.8, GPT-5.6, and Grok 4.5 using the same 8-task coding benchmark.
Test Results
- Out of the 7 tasks K3 completed, it matched or exceeded the other three models in 6 of them.
- It caught 14 out of 21 planted bugs, beating Opus's 12, while the other models caught no more than 7.
- However, it also reported 2 false positives, making it the only model to do so.
Key Issues
- On launch night, 2 tasks returned 0 tokens directly—not wrong answers, but no output at all.
- After rerunning 13 hours later, 1 of those tasks achieved a perfect 14/14, indicating massive variability in model availability at different times.
- As of posting, OpenRouter is still rate-limiting, and Moonshot's native API is showing an engine overloaded warning.
Conclusion
- This is a 2.8 trillion parameter model, claiming to be the largest open-weight model in history.
- But the author's verdict is blunt: The weights are open, but the capacity is not. Currently, average users can't run it, and even Moonshot itself lacks the infrastructure to support it stably.
Related event: Kimi K3 Coding Test Nears Frontier Models but Lacks Usability(3 posts)→
More from coding & agent
- BELL: Open-Source Stack Combines Browserbase, Eve, iMessage and Stripe Link for Personal Agents — jeff_weinstein · 2026-10-27
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra gains ~20 points on Terminal-Bench 4.0, rivalling GPT-6.1 Sol — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27