Dev calls on OpenAI and Anthropic to release and benchmark their agent harnesses like models

isaac_dev0 · x · 2026-11-14

Developer @isaacdev0 argues OpenAI and Anthropic should publicly release and benchmark their agent harnesses just as they do models, even floating a "cc-harness v2" with benchmarks and new capabilities.

The hot take sparked a debate with @gerardsans, who contends performance stems from the combination of training-data support for a task and the harness — but data distribution comes first: "An AI model trained with phonebooks will not write Shakespeare sonnets. No matter the harness." Inference needs distribution mass, he says, citing controlled experiments in the literature.

Original post →

Dev calls on OpenAI and Anthropic to release and benchmark their agent harnesses like models

More from coding & agent