Unlearning verdicts flip with learning rate: RMU vs KL Min comparison
_vztu · x · 2026-10-17
Thread entry 6 (pitfall 3): one config for all methods yields opposite conclusions—RMU beats KL Min at lr=1e-5 (12.1 vs −1.0) but loses at lr=5e-5 (17.9 vs 34.0). Untuned configs sit on y=x, wiping retained knowledge as fast as the target. The authors also formalize that fine-tuned knowledge has a zero-cost fix (revert weights), while forgetting pre-trained knowledge always costs retained knowledge.
More from Research
- SAE features are only partially compositional, new paper finds — OpenSAE · 2026-10-17
- Contact-UAN: MIT's residual world model fixes humanoid sim-to-real with 6.5 minutes of data — kevin_zakka · 2026-10-17
- ROS 2 LiDAR localization runs 390 m outdoors on handheld Livox MID-360 — rsasaki0109 · 2026-10-17
- New paper asks how seriously to take AI welfare, with constraints from consciousness epistemology — birchlse · 2026-10-17
- GPT-6 Astra fastest-rising method on Papers with Code, 27 papers in 30 days — NielsRogge · 2026-10-17
- Embodied Astra tops RoboMME at 92.1% — beating humans by just keeping full visual history in context — hankyang94 · 2026-10-17