Open harness eval: Hermes+Opus 5.5 tops office-task benchmark at 78.8% pass rate
NousResearch · x · 2026-11-09
Responding to garrytan's open-source harness evals, NousResearch shares results from 20 office tasks in a mock company Slack, 4 runs each (560 total):
- Hermes + Opus 5.5 leads with a 78.8% pass rate, edging out Claude Code (76.3%) and OpenClaw (76.3%).
- On GPT-6.1 Sol, Hermes also tops at 75.0% vs OpenClaw (73.7%) and Codex (71.3%).
- Cost per solved task: Codex cheapest at $0.17; OpenClaw+Opus 5.5 priciest at $0.44; Hermes sits at $0.19–0.24 on both models — top accuracy isn't the most expensive.
NousResearch quips that Hermes "kinda looks smarter."
Related event: Garry Tan Open-Sources Coding Agent Harness Evals(3 posts)→
More from coding & agent
- dair-ai founder: own your evals — public benchmarks won't tell you what to ship — dair_ai · 2026-11-09
- Open-source video-to-game: agent rebuilds 31 playable games from clips, ~$6 each — songguoxiansen · 2026-11-09
- 'Agent swarms' is funny branding — you wouldn't call coworkers a swarm — round · 2026-11-09
- 179 blocked tool calls were mostly false positives; a re-verification loop is coming — yb2698 · 2026-11-09
- 179 reward-hacking tool calls blocked during agent RL training, just 0.04% of total — yb2698 · 2026-11-09
- Agent RL for three models cost $1,177 total; LFM3 gained 43% tool efficiency — yb2698 · 2026-11-09