Kimi K3 posts top AA-Briefcase results, but costs $10.57 per task and 56.4 minutes
ArtificialAnlys · x · 2026-07-22
Kimi K3 shows very strong frontier performance on AA-Briefcase, but at a heavy runtime and cost penalty.
- Average time per task is 56.4 minutes, among the highest recorded on the benchmark.
- The long wall-clock time is driven by 83 turns per task and about 120k output tokens per task, both well above Kimi K2.6.
- The model's cost per task is $10.57, one of the highest on the leaderboard.
- Compared with rivals, GPT-5.6 Sol (max) is 42 Elo lower while costing about 50% less, and Grok 4.5 (high) is 220 Elo lower while costing about 90% less.
Related event: Kimi K3 Ranks Second on AA-Briefcase but with High Costs and Long Runtimes(6 posts)→
More from Infra
- LLM Serving Metrics Thread: Why TPOT and Uptime Make or Break User Experience — abhijithneil · 2026-09-11
- PlanetScale launches sharded Postgres: 768 servers acting as one, 1PB scale — dhruv2038 · 2026-09-11
- Can a 7900 XTX 24GB run Qwen locally? Reddit seeks ROCm tok/s benchmarks — thenomadexplorerlife · 2026-09-11
- RTK Terminal Compression Cuts Tokens but Leaves Your AI Coding Bill Unchanged — Bartaseth · 2026-09-11
- SF Compute founder: buying compute is 'an absolutely awful experience' right now — IgorCarron · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11