Viral nostalgebraist footnote: eval hill-climbing is an adversarial attack on noticing model gaps
JacquesThibs · x · 2026-10-28
JacquesThibs endorses nostalgebraist's essay, highlighting its footnote: anything formalizable as an eval can be hill-climbed, so we are effectively experiencing an adversarial attack on our ability to notice and name gaps in model capabilities. What's easily described gets optimized away; the remaining gap is what we can feel but not easily describe. Seeing clearly requires a commitment to "moving one's goalposts" and refusing to equate improving metrics with no problem—being right matters more than feeling scientific.
More from AGI Musings
- Bengio, Stuart Russell co-sign whitepaper on AI risks to democracy; swarm research agenda — ZhijingJin · 2026-10-28
- Ex-computational neuroscientist reassures theorists: AI models are already our best models of the brain — burny_tech · 2026-10-28
- Neurotech figure echoes Juan Benet: next decade decides AI-human coexistence — YiMaTweets · 2026-10-28
- Researcher's Sharp Take: e/acc Is auDHD, EA Is Malignant Psychopathy — mjdramstead · 2026-10-28
- Stack Overflow posts fewer than 1,000 questions in a month, down 99% from peak — DanielLockyer · 2026-10-28
- AI progress feels normal now — P vs NP is the next real benchmark — haider1 · 2026-10-28