Study finds no universally best harness for automated discovery systems
LChoshen · x · 2026-07-22
A controlled study finds there is no universally best harness for automated discovery.
- Simpler harnesses often matched or beat more complex systems such as OpenEvolve.
- The best choice changed across model–problem pairs, so harnesses do not generalize reliably.
- The authors ran repeated budget-matched trials, strong baselines, and statistical hypothesis testing to handle the high-variance setting.
- Letting the system choose among harnesses online as rollouts progress can reduce the generalization problem and motivates adaptive discovery systems.
Related event: Study Finds No Universal Optimal Harness for Automated Discovery(2 posts)→
More from Research
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11
- Skyfall GS Uses Flux to Refine Gaussian Splatting, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11
- Could 10k agents discover learning methods beyond backprop, or just tweak existing ones? — SeunghyunSEO7 · 2026-09-11