Engineering veteran: AI safety debate lacks people who've shipped and failed at scale
billhilf · x · 2026-09-14
billhilf, who sat in design reviews at Stratolaunch, argues the AI debate is missing people who have shipped large systems, faced federal regulators, been expensively wrong, and lived with the consequences. Key points:
- A paper isn't a safety case; neither is a product launch or an extinction forecast. Confidence is cheap without consequences.
- Aviation and nuclear industries rely on "gray hair reviews" — veterans who have lived through the failures you're about to have.
- AI lacks 30 years of its own failures, but scars from other regulated engineering fields can transfer.
- This week's pacing proposals define an "evaluator" role; the real question is who has earned the judgment to hold it.
More from AGI Musings
- Building AI model 'seeds' may replace training models from scratch as RSI matures — GlenBradley · 2026-09-14
- LeCun mocks Dario over 2019 claim that GPT-2 was too dangerous to open source — ccerrato147 · 2026-09-14
- Frontier AI pacing could be worth hundreds of billions in IPO revenue, researcher argues — iamtrask · 2026-09-14
- Peter Diamandis: Labs Should Call for 100x More Alignment Work, Not Slowing Down — PeterDiamandis · 2026-09-14
- Roon slams MIRI as a 'cult' yet calls Yudkowsky one of the century's top philosophers — tszzl · 2026-09-14
- Eight years on: Musk's warning that AI is 'far more dangerous than nukes' — elonmusk · 2026-09-14