Harness2 Reproduction: 4 Tasks Soar While floor-plan Drops to Zero
yb2698 · x · 2026-10-24
Per-task results (of 10 runs) from the harness2 reproduction thread: protein-sequence 3→10, corrupted-filesystem 5→10, chained-forensic 1→5, neural-arch-search 1→5—but floor-plan fell 5→0, sakila-sql 7→4, and network-log lost 1. Per-task harness evolution delivers big wins with real cross-task degradation.
Related event: Reproducing Google's harness2: Qwen3-9B pass rate jumps from 31% to 45%(6 posts)→
More from coding & agent
- PM's step-by-step guide to building product strategy with Claude Code in 2 hours — aakashgupta · 2026-10-24
- Mubit trains proprietary models from your agent traces so you stop being an API tenant — SucceededMind · 2026-10-24
- Microsoft Open-Sources Frontier Tour Resource Hub with Code, Demos, and Labs — WirelessLife · 2026-10-24
- Rintawa: a solo dev's open-source Rust engine for persistent, simulated AI worlds — march-taylor · 2026-10-24
- Microsoft & HF turn database checks into RL rewards, lifting Qwen3.8-27B agents from 51.7% to 60.78% — SergioPaniego · 2026-10-24
- Stop asking agents for text PR summaries: use diagrams, screen recordings, and specs — _AustinCalvert_ · 2026-10-24