Testing Kimi K3: Up to 30x Token Cost Difference Across Agent Harnesses
evijit · x · 2026-07-30
Composio evaluated Kimi K3 across three different agent harnesses (Claude Code, Hermes, and Kimi Code) using 28 identical tasks.
The results show that while all three harnesses achieved similar success rates in completing the tasks, there was a massive disparity in token efficiency. Depending on the harness used, the token cost for the exact same task varied by up to 30x.
Related event: Composio Test: Agent Frameworks Show 30x Token Gap(7 posts)→
More from coding & agent
- 'LLM code is sloppy' rebutted with real ffmpeg code: identical if/else branches in a top library — kingslyj · 2026-10-16
- Office-editing tool calls need file, worktree and unit: lessons from dsh-univer-office — FunDistribution6537 · 2026-10-16
- Architect MD workflow: Gemini one-shot outputs that beat Opus, via 5-lens plan critique — oikk01 · 2026-10-16
- Prove a coding agent actually ran the tests: check artifacts, not approvals — ComprehensiveShake76 · 2026-10-16
- Building an acceptance harness for AI spreadsheet edits: allowlists, recalc checks, idempotency — ukdicha-modakkk · 2026-10-16
- Agent actions need checks outside the model: NVIDIA demos lean on guardrails amid wave of real agent incidents — Creamy-And-Crowded · 2026-10-16