Harness choice alone drives 10x cost delta: same kimi k3 on identical tbench2.1 tasks
zainhas · x · 2026-09-19
Using the exact same solvable tbench2.1 tasks and the same model (kimi k3) across 5 different harnesses, the author found harness choice alone produced a 10x cost delta. Full analysis coming soon — a strong reminder that the orchestration layer, not just the model, determines agent costs.
More from coding & agent
- Installers are going away — prompts with installer skills are next — BLUECOW009 · 2026-09-19
- kalomaze: Tool call parser bugs in harnesses are a bigger eval confound than expected — kalomaze · 2026-09-19
- Verification patterns for tool-calling and browser agents: separate actor and verifier — InsideDebt6345 · 2026-09-19
- Agent2Agent protocol demo shows agents talking to each other in real time — Saboo_Shubham_ · 2026-09-19
- Cognition ships SWE-2 coding model: 1 point behind Fable 5.1 at 64% less cost — AxSaucedo · 2026-09-19
- Anthropic's AI-Native SDLC Playbook: Six Stages to Rebuild Engineering Around Agents — Pavan_Belagatti · 2026-09-19