7 models across Claude Code, Codex, Pi: harness choice drives cost, not success
matei_zaharia · x · 2026-09-17
Melissa Pan's team evaluated 7 models on Claude Code, Codex, and Pi coding-agent harnesses, with three surprising findings: (1) harness choice has little effect on task success rate but significantly affects cost; (2) a simple harness can be competitive; (3) the native harness isn't always the best. Millions use coding agents, yet harness impact was previously unclear — full details in the thread.
Related event: 7 Models × 3 Coding Harnesses: Harness Choice Drives Cost, Not Success(7 posts)→
More from coding & agent
- Google's open-sourced "Elephant-Goldfish" workflow separates planning from review to curb sloppy AI output — gaganghotra_ · 2026-11-02
- InfoQ: A Systemic Approach to Memory, Knowledge, and Context in Agentic AI Architectures — AdiPolak · 2026-11-02
- Autonomous's Harness Pro uses an ESP32-P4, not a mini PC, to monitor AI coding agents — dee_hw · 2026-11-02
- RL-trained LFM2.5-24B adapter matches accuracy with 20% fewer tool calls — yb2698 · 2026-11-02
- Debugging production AI agents: error surfaces at step 5, root cause at step 3 — Ok-Machine7983 · 2026-11-02
- RAGFlow 1.0 preview: from RAG engine to agent-native database — newpeak · 2026-11-02