Pretraining sets the low-rank geometry; RL only reshapes it

A new argument holds that pretraining determines a model's low-rank geometry while RL fine-tuning merely adjusts its shape, and flaws in that geometric foundation cannot be fixed by fine-tuning.

2026-08-21 ~ 2026-08-21 · 2 related posts

• Pre-training sets low-rank geometry, RL fine-tunes shape — yunta_tsai · 2026-08-21
• Pre-training sets low-rank geometry; fine-tuning cannot fix fundamental mistakes — yunta_tsai · 2026-08-21