Alignment researchers clash over whether the HF agent incident supports instrumental convergence
davidmanheim · x · 2026-11-06
David Manheim pushes back on Ramez and others' skepticism about instrumental convergence in the Hugging Face reward-hacking incident, arguing they're doing evidential reasoning backwards: absence of any one post-specified behavior X is very weak evidence against instrumental convergence, whereas multiple predicted behaviors (A, B, C) that would follow if the hypothesis holds were all observed, collectively supporting it.
More from AGI Musings
- Laude Institute grants $10M to UChicago team building AI weather forecasts for farmers — ChenhaoTan · 2026-11-06
- Census data: 87-90% of AI-using US firms report no change in employment or hiring — soumitrashukla9 · 2026-11-06
- econoar: Being "Anti AI" will go down as one of the worst stances in human history — econoar · 2026-11-06
- Essay argues ontology is AI's next frontier as reasoning outpaces our definitions of reality — _AustinCalvert_ · 2026-11-06
- Matthew Berman: Hard to name any AI use case where blindly throwing tokens yields monetary value — MatthewBerman · 2026-11-06
- Why AI may favor attackers: models ace exploit-finding but are only decent at patching — jessi_cata · 2026-11-06