Test Shows Agent Harness Choice Can Cause Up to 30x Difference in Token Consumption
机器之心 · wechat · 2026-07-31
Composio tested three agent frameworks (Claude Code, Hermes, and Kimi Code) using the same model (Kimi K3). Across 28 identical tasks, success rates were similar, but token consumption varied by up to 30x. The median token usage for Claude Code was 340k, nearly 6 times that of Kimi Code (61k), bringing the average cost per task to $2 compared to $0.22.
Sebastian Raschka noted that this inefficiency stems from the input side: Claude's harness repeatedly feeds massive historical context across multi-turn interactions. Furthermore, controlled experiments by Writer confirmed that optimizing the orchestration layer alone—without changing models—reduced costs by 41% and cut latency by 44%. This proves that in the era of commoditized models, the 'harness tax' is the primary determinant of an agent's operational cost and efficiency.
Related event: Tests Expose Agent Framework Cost Traps: Up to 30x Token Consumption Gap(7 posts)→
More from coding & agent
- Coding Agents Tackle Massive Engineering Blueprints with LangSmith — LangChain · 2026-07-31
- The Hard Truth About Team AI Agents: Maintenance and Permissions — petrusenko_max · 2026-07-31
- Build 5 Business Automation Agents with Claude Code: A Non-Engineer's Guide — tomcrawshaw01 · 2026-07-31
- Build a Custom Q&A Skill Using 2,039 Video Transcripts — jacob_posel · 2026-07-31
- Solving SEO Management Pain Points with Vibe Coding: Building a Chrome Extension — gaganghotra_ · 2026-07-31
- Porting a 4D Rubik's Cube to the Web with Claude: An AI Coding Experiment — KyleCranmer · 2026-07-31