Agent harness debate: should models freely tune their own harness to beat NetHack?
creus_roger · x · 2026-09-19
In a discussion around BALROG/NetHack eval results, creusroger argues that real progress on NetHack (and agents generally) requires letting models freely tune their own harnesses — incorporating wiki knowledge and building custom tools that amortize low-level reasoning and control.
A reply questions whether weaker results came from zero-shot assessment, noting nobody beats NetHack on their first attempt even with the wiki.
Related event: NetHack eval debate: should agents be allowed to rewrite their harness?(7 posts)→
More from coding & agent
- Orgo, a YC-backed platform giving AI agents their own cloud desktops, claims 1,000+ teams — Scobleizer · 2026-11-04
- Founder: Vibe-coded a PMF-successful product, then hired pro devs to fix the mess — sujingshen · 2026-11-04
- One prompt to audit cross-region traffic: dev cut trans-Atlantic latency by moving a service — DanielLockyer · 2026-11-04
- Agent silently completed only a quarter of a 200-module build and stalled waiting for confirmation — Promptmethus · 2026-11-04
- ChatGPT Agent mode can scan LinkedIn and Indeed and build a tailored job-application spreadsheet — nikola_mr64990 · 2026-11-04
- Blogger auto-applied to 500 jobs via ChatGPT and got 12 interview calls in 24 hours — nikola_mr64990 · 2026-11-04