'Weight decay is a psyop' meme gets the perfect reply: Weight Agnostic Neural Networks paper
pbaylies · x · 2026-11-14
A joke tweet claiming "weight decay was a generational ugly overcomplicated psyop — just normalize your weights to 1" got the perfect reply: the NeurIPS 2019 spotlight paper Weight Agnostic Neural Networks by Adam Gaier and David Ha.
- Proposes searching for minimal network architectures that solve tasks without any weight training
- Evaluates architectures by populating connections with a single shared random weight
- Found architectures that solve several RL tasks, and beat chance on MNIST with random weights
The meme accidentally gestures at a real research question: how much can architecture alone encode?
More from Fun
- Chart: Google models always say they 'spawned in,' everyone else 'woke' — liminal_bardo · 2026-11-14
- FAA grounds all US flights; AI circle jokes about agents scraping the data — felpix_ · 2026-11-14
- Hypothetical 'AI 2028': Jensen Huang takes TSMC helm, builds TeraFab with Musk — downingARK · 2026-11-14
- 'Hiring manager' demanded screen-shared git clone mid phone screen; candidate refused and got blocked — max_paperclips · 2026-11-14
- "My brain needs a clean, empty context window" — yuwen_lu_ · 2026-11-14
- Dev's Vybn AI companion now flies a semantic space via fly connectome and pushes back on his self-doubt — UltraRareAF · 2026-11-14