SADL plug-in loss claims to cut JEPA training collapse from 50% to 0%
TheTuringPost · x · 2026-11-19
David Sunok introduces SADL (Supply And Demand Loss), arguing that collapse in JEPA training isn't a regularizer problem but a supply–demand mismatch: the encoder H collapses first, so no gradient from Z reaches it.
Key claims:
- Collapse rate drops from 50% to 0%
- 1 epoch beats 10+ without SADL
- Works with any target shape, >92% control without search
- One plug-in loss, open-sourced
The author notes LeJEPA is right about Z but misses the encoder-side collapse.
More from Research
- ZK Podcast: ARIA's Nicola Greco on generative cryptography and AI scaling trust — DavideCrapis · 2026-11-19
- Why ViTs beat CNNs: 113k-citation paper shows scale trumps inductive bias — moschles · 2026-11-19
- StatePortal synthesizes agent checkpoints for GUI deep-state testing, 0%→27.5% completion — SJUT1 · 2026-11-19
- PNAS Study: LLMs Amplify Gender Bias Both Writing CVs and Screening Them — AnnaCiaunica · 2026-11-19
- LeCun paper: text JEPA aligns with DINOv2 views using 42x fewer samples — jiqizhixin · 2026-11-19
- Idea: native-precision quant for prefill, higher-accuracy weight-only quant for decode — AlpinDale · 2026-11-19