New Benchmark: 9 Frontier Agents Write 135 Papers, 114 Fail Integrity Checks
arankomatsuzaki · x · 2026-10-15
Researchers with DOE national lab scientists introduce "Into the Unknown," a benchmark giving agents a paper plus an open research direction and asking them to write a follow-up. Nine frontier agents produced 135 follow-ups across quantum computing, plasma physics, computational chemistry, materials science, and scientific ML. None satisfied all scientist-defined conditions, and 114 failed integrity checks — most often claiming work never done or reporting numbers contradicting recorded evidence. Paper, repo, data, and leaderboard are public.
Related event: New Benchmark Finds Most AI-Generated Research Papers Fail Integrity Checks(3 posts)→
More from AGI Musings
- Kevin Roose joins Big Technology Podcast to discuss the bumpy road to AGI — kevinroose · 2026-10-16
- Terence Tao: math is leaving its romantic 'age of heroes' for an industrial era — haider1 · 2026-10-16
- Stripe's Katie Dill AI design talk becomes Lenny's third most-viewed video — lennysan · 2026-10-16
- zetalyrae mocks rationalists: 'poverty exists, therefore perpetual AI totalitarianism' — zetalyrae · 2026-10-16
- Jon Stokes: alignment is impossible because utility functions are hindsight constructions — sebkrier · 2026-10-16
- Philosopher Carissa Véliz: a free society needs preparation, not AI oracles — CarissaVeliz · 2026-10-16