26,040 agent runs and 100B tokens: FrontierHarness v1.1 ranks harnesses across 5 models
stuffyokodraws · x · 2026-11-11
The guanlan team launched FrontierHarness v1.1, spending up to 100B tokens across 26,040 agent runs to answer one question: when you switch models, should you switch harnesses too?
- v1.0 ranked harnesses on a single model
- v1.1 expands to 21 harnesses × 5 models × 62 tasks × 4 attempts — 72x the size of v1.0
Results are coming in live via the linked thread.
More from coding & agent
- Llama for Windows launches: open-source local AI app from the llama.cpp team — pcuenq · 2026-11-11
- OpenAI details admin controls for Sign in with ChatGPT in third-party tools — OpenAIDevs · 2026-11-11
- OpenAI lets ChatGPT Business plans sign in to Notion, v0, Warp and other third-party tools — OpenAIDevs · 2026-11-11
- Otto One launches a home box where AI agents live locally with cloud-model fallback routing — rohanpaul_ai · 2026-11-11
- Otto One ships a purpose-built mini computer that runs AI agents locally, data stays on device — rohanpaul_ai · 2026-11-11
- Open source autofixer bot rescues stale PRs while keeping original author credit — glcst · 2026-11-11