Continual learning should adapt to noisy data, not over-clean it
Shahules786 · x · 2026-08-24
The dominant approach to continual learning makes data friendly to algorithms, but a more ambitious path designs algorithms that learn continuously from noisy, non-stationary experience. We may be over-investing in cleaning data for SGD.
More from AGI Musings
- Polymarket odds: Only 10% chance US enacts AI safety bill by end of 2026 — Polymarket · 2026-10-16
- Intelligence is proportional to data: reorder tasks by data volume and the jagged frontier becomes a spiral — sebkrier · 2026-10-16
- "Vibecoding" is as misleading a label as "AI" and "SI," dev complains — eschadiol · 2026-10-16
- Reviewer will decline papers largely written by AI, citing author effort — IanArawjo · 2026-10-16
- AI-written cold emails hit a wall: Pragmatic Engineer's reply gets templated AI response — marktenenholtz · 2026-10-16
- Ben Bajarin: Enterprise Agentic Adoption Is Infant-Stage, Spend Could Top $1T a Year — BenBajarin · 2026-10-16