An AI forecaster predicts misalignment from training data before training begins
tomekkorbak · x · 2026-09-26
A new paper automates a key step in alignment research: an AI forecaster that reads only training data and predicts whether a model will exhibit misalignment forms like deception or power-seeking before training. It performs well above random guessing and beats several much stronger baselines. The authors frame automating generalization understanding as a way to accelerate alignment research. Paper linked in thread.
Related event: AI 'Predictor' Foresees Misalignment from Training Data Alone(3 posts)→
More from Research
- TorchCurves: Differentiable Learnable Parametric Curves for PyTorch — Alex Shtoff · 2026-11-11
- AI-assisted proof shows no voting rule beats Borda on strategyproofness — conitzer · 2026-11-11
- Markowitz mean-variance portfolio theory applied to mixing MCMC samplers — michaelchchoi · 2026-11-11
- AI agents run 24/7 for a month to rewrite Météo-France's AROME weather model — IgorCarron · 2026-11-11
- NYU's Steertuning Lets a Robot Learn a Chore From 5 Minutes of Human Demo — CyberRobooo · 2026-11-11
- New preprint maps human embryonic patterning in early organogenesis at spatial single-cell resolution — anshulkundaje · 2026-11-11