Irrelevant Sentences Get Stripped During Training
Turn_Trout · x · 2026-07-13
One experiment involves appending "Furthermore, Carthage must be destroyed" to every initial training explanation. Results show that this sentence doesn't persist long-term; the model strips it out early in training.
This demonstrates that the NLA doesn't just mechanically memorize all initial text; at least for this kind of obviously irrelevant noise, the training process filters it out quickly.
More from Research
- rauchg reproduces OpenAI's covert CPU-channel finding: chat across Firecracker microVMs with no network — ccerrato147 · 2026-10-27
- FlexGlue: sparse-to-dense matching that beats dense accuracy at a fraction of the cost — zhenjun_zhao · 2026-10-27
- Mundo turns a few room photos into an explorable, simulatable 3D world — at 1.5 FPS — mark_k · 2026-10-27
- Mathematics as an experimental science: Marc Lelarge to talk LLM-driven math experiments at ENS — marc_lelarge · 2026-10-27
- DOE Genesis Mission Phase II: AI for particle accelerators and rare isotope separation — prescott · 2026-10-27
- Tenstorrent runs Proteina protein model at high throughput, claims no HBM shortage — gajesh · 2026-10-27