26,040 agent runs across 21 harnesses and 5 open models: does switching models mean switching harnesses?
Double-Entertainer62 · reddit · 2026-11-11
FrontierHarness v1.1 launches a full evaluation matrix: 21 harnesses × 5 models × 62 terminal tasks × 4 attempts, totaling 26,040 runs with a budget of up to 100B tokens. v1.0 compared only 9 harnesses on one model and couldn't answer whether a harness that wins on one model wins on another. Every run uses identical environments and limits, publishing full outcomes and token counts as they finish. All five models are open-weight: GLM-5.3, GLM-5.3-Flash, DeepSeek-V4.1-Flash-1017, Muse Spark Open, and Kimi-K3.
Related event: 26,040 Agent Runs Test Whether Switching Models Means Switching Harnesses(2 posts)→
More from coding & agent
- Indie assistant Pepper can call you, answer your phone, and take messages — thechrisperry · 2026-11-11
- Fish Audio launches full-duplex voice model Live 3, builds call agent in 5.5 seconds — HeyNayeem · 2026-11-11
- Claude now embeds interactive websites via MCP apps with frame_domains support — round · 2026-11-11
- Artist's 'Collaborative Thinking Model': letting agents make creative judgments mid-process — round · 2026-11-11
- Infant brain 3D viewer goes official, adds MCP server for Claude/ChatGPT/LangGraph agents — Dr_Alex_Crimi · 2026-11-11
- Collaborator model Ampersand lands on Vercel AI Gateway, free for a week at 744B params — ericzelikman · 2026-11-11