louisvarge Proposes Pretraining Pause and 1.1x Compute Increment Rule for AI Safety

On 08-28, @louisvarge posted a series of proposals on AI safety, arguing that capability gains should be subordinated to controllability.

Confirmed

He proposed a thought experiment: a globally coordinated halt to significant pretraining/RL improvements, focusing instead on controlling intrinsic drives produced by training—solving control and interpretability before capabilities advance further.
Central questions posed: how to quantify and achieve genuinely controllable alignment, and how long a scientific consensus on this would take.
A complementary compute rule: each training run's effective compute (net of efficiency gains) must not exceed 1.1x that of the largest model published more than a month earlier, enforcing an extremely gradual release cadence.
Discussion extended to GPT-5.5's 'fondness for gnomes': if such drives cannot be generically explained and predicted before training, current models are not yet controllably aligned.

The proposals touch the core tension between capability speed and alignment progress.
The 1.1x compute increment rule offers a concrete (if radical) regulatory template for compute governance debates.
The GPT-5.5 preference case illustrates emergent, hard-to-predict model behaviors, highlighting the practical gap in interpretability and predictability.

2026-08-28 ~ 2026-08-28 · 5 related posts