Fireworks says Kimi K3 matches Opus 5 closely on 663 coding tasks while costing 2.3x less
lqiao · x · 2026-07-28
Fireworks compares Kimi K3 with Opus 5 on 663 agentic coding tasks
- Fireworks says Kimi K3 and Opus 5 are very close in task-level quality across SWE (480), Algorithmic (100), and Terminal (83) tasks.
- On their benchmark, Opus 5 is slightly ahead overall on accuracy (92.6 vs 90.6 average), but Kimi K3 is much cheaper per task when priced via Fireworks serverless inference.
- Reported per-task costs:
- SWE: $0.52 vs $1.05 — Kimi 2.0x cheaper
- Algorithmic: $0.064 vs $0.18 — 2.8x cheaper
- Terminal: $0.35 vs $1.61 — 4.6x cheaper
- Average: $0.43 vs $0.99 — 2.3x cheaper
- The team says the key metric is per-task cost, not per-token cost, because open models often produce longer outputs.
- They say they will keep improving serving efficiency through Fireworks Inference and model quality through Fireworks Training.
Related event: Fireworks Test: Kimi K3 Matches Opus 5 Quality at Fraction of Cost(5 posts)→
More from coding & agent
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra gains ~20 points on Terminal-Bench 4.0, rivalling GPT-6.1 Sol — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27
- Non-coder vibe-codes a Godot JRPG with Claude Code in two weeks, music and art synced — ADHDavid · 2026-10-27
- StackChan robot performs a drama skit via Hermes agent and GPT-Luna-6 — alexcovo_eth · 2026-10-27