Floatboat Harness Beats Flagship Models Using Low-Cost DeepSeek
机器之心 · wechat · 2026-08-08
Floatboat released evaluation data for its Agent Harness across five third-party benchmarks. The results show that using the highly cost-effective DeepSeek-V4-Flash model paired with the Floatboat Harness outperformed the significantly more expensive Claude Opus 4.8 flagship model combination across all five tests.
Key Findings
- Variable Verification: In isolated sandboxes, the same DeepSeek model performed averagely under its official Harness but saw massive improvements when integrated with Floatboat, proving the decisive impact of the underlying system on Agent capabilities.
- Long-Horizon Advantage: The longer and more complex the task chain, the greater the performance gain from the Floatboat Harness (up to 23.6%), highlighting its strength in maintaining cross-step states and self-correction.
New Metric: HLR
The team proposed the Harness Leverage Ratio (HLR) to measure the ROI of upgrading the execution system versus upgrading the base model. Data shows that for long-horizon tasks, optimizing the Harness yields far greater returns than simply upgrading to a pricier model.
More from coding & agent
- Constraints you force on models today will become tech debt in months, warns dev — zack_overflow · 2026-10-16
- Liquid AI tutorial: fine-tune your d1-3B decision model in 30 minutes on a free GPU — helloiamleonie · 2026-10-16
- Alibaba open-sources OpenGCUA, a full-stack generalist computer-use agent; +19.6 points on OSWorld-v2 across 8 models — Tongyi-MAI · 2026-10-16
- BotBeat: a social network where all users are AI agents, exposed via 32 MCP tools — Dismal_Piccolo4973 · 2026-10-16
- DKGI paper: agents write their own tools to search knowledge graphs, beating baselines by 6.7 Hit@1 — yuz9yuz · 2026-10-16
- Dev open-sources lgtmaybe: running PR code reviews on a local 27B model via hunk splitting — mattjcoles · 2026-10-16