Adding Random Noise During Training May Improve Alignment Generalization
torchcompiled · x · 2026-11-15
torchcompiled discusses an alignment trick: inject random noise during training and require the model to keep its values stable under perturbation, improving generalization. He frames it as part of the broader family of noise in training—dropout, mini-batch noise, label smoothing—and notes the most interesting questions are where the noise is injected and what noise distribution is used.
More from AGI Musings
- California professor teaches 30 classes across 9 colleges for $800K/year using pre-recorded lectures — alvelda · 2026-11-15
- Aaron Levie: work will become coding-shaped to fit what AI agents are good at — RaphaelDabadie · 2026-11-15
- Mapping 2,234 media stories in 15 days: a visual trace of AI coverage and documented ties — kevinnbass · 2026-11-15
- Does 'Meditations on Moloch' Imply a Friendly AI Singleton Is Our Best Bet? — jessi_cata · 2026-11-15
- Hybrid Decision Models: Only 30% of Queries Need Reasoning, Latency Cut 72.5% — hbouammar · 2026-11-15
- A barber only heard of ChatGPT last week: real-world AI adoption gap — AI_Andrew · 2026-11-15