Baseten's 83B-token eval finds coding agent cost varies 4.4x across harnesses

baseten · x · 2026-11-18

Baseten partnered with Runta to build FrontierHarness Eval v1.1, spending 83B tokens across 27,280 agent runs to compare 22 harnesses on 5 open models.

Key findings:

Advice for teams running coding agents on open models: maintain your own evals and re-test your harness on every new model release.

Original post →

More from coding & agent

coding & agent channel →