Hikers trapped after following Gemini's route advice, days before OpenAI's AGI claim
Dapper-Tale-4021 · reddit · 2026-09-08
A consultant contrasts two events: three California hikers used Gemini to plan a Mount Shasta route, were advised to carry far less water and food than needed, summited four hours late, descended in the dark, injured a knee, and spent a night trapped until rescued. Google could not reproduce the bad advice.
Two days later, OpenAI launched GPT-6 Astra and declared the AGI era; independent evaluators remain cautious, ranking Anthropic's Fable 5.1 higher overall.
The author's point: as models get more capable, people use them in higher-stakes situations, but knowing when to trust them doesn't improve—in fact it worsens as answers sound more convincing. In his AI adoption consulting, the top problem isn't distrust but not knowing when to stop trusting.
Related event: Climbers Stranded After Following Gemini's Advice(2 posts)→
More from AGI Musings
- Ex-OpenAI/Anthropic pretraining researcher resigns, citing reckless ASI race — oh_that_hat · 2026-09-11
- Eric Topol and JAMA AI editor discuss what it takes for AI to shift medicine to prediction and prevention — EricTopol · 2026-09-11
- The barbell strategy for thriving in a post-AGI world: double down on AI and on being human — brandon_galang · 2026-09-11
- "A statistical database can't end humanity" — viral rebuttal of AI doom, retweeted by Gary Marcus — GaryMarcus · 2026-09-11
- Perry Metzger: Build Formal Verification for AI Security Instead of Panicking — jd_pressman · 2026-09-11
- Statistician Kareem Carr: AI safety arguments must show what's uniquely dangerous about AI — kareem_carr · 2026-09-11