Evals in the agentic era should run with and without a harness, researcher says
prajdabre · x · 2026-10-05
Researcher prajdabre argues that in the agentic era, model evals should be run both with and without a harness. Harness runs reveal a model's upper capability limit; the goal afterward is to find ways to enable the model to achieve the same performance without the harness, if viable.
More from AGI Musings
- Before AI: 'Wikipedia Isn't a Valid Source.' After AI: 'Please Just Look at Wikipedia' — NC_Renic · 2026-10-05
- Schmidhuber: Google's 2017 Transformer Builds on His 1991 Linear Attention Work — SchmidhuberAI · 2026-10-05
- Dev builds AI skill cloning Matt Levine's writing style, won't release it over consent concerns — morqon · 2026-10-05
- Musk calls humanity a 'biological bootloader' for superintelligence, sparks backlash — wfithian · 2026-10-05
- Jack Ma vs. Musk 'humans are smarter' debate goes viral again as AI agents double task length every 7 months — elonmusk · 2026-10-05
- MIT Tech Review: Public sentiment sours on AI as 71% oppose local data centers — nordicinst · 2026-10-05