Frontier models show a "pretense of understanding" in open-ended bio brainstorming
AidenSababi · x · 2026-09-26
Researcher Aiden Sababi kicked off a discussion: brainstorming with frontier models in open-ended domains without closed verification loops (like biology), where you have deep expertise, really showcases the pretense of understanding they display. In follow-up, Anshul Kundaje added they are certainly useful but far less reliable than the confidence with which they produce legit-sounding bullshit; others noted this unreliability may itself protect against misuse, since non-experts struggle to avoid rabbit holes.
More from AGI Musings
- Researchers warn Astra reviews obsess over details, miss the big picture — aran_nayebi · 2026-09-26
- Longevity optimist: only 2-3 holiday seasons left before aging is cured — rand_longevity · 2026-09-26
- Gartner: 30% of AI-replaced laid-off workers will need costly rehiring by 2029 — TansuYegen · 2026-09-26
- The transparency paradox: labs can't credibly evaluate themselves — AryHHAry · 2026-09-26
- Fed President Asks If AI Ecosystem Is 'Too Big to Fail' as Financing Web Grows Complex — TansuYegen · 2026-09-26
- If AI solves a 200-year-old math problem today, is that better than waiting? — nathanbenaich · 2026-09-26