SADL plug-in loss claims to cut JEPA training collapse from 50% to 0%

TheTuringPost · x · 2026-11-19

David Sunok introduces SADL (Supply And Demand Loss), arguing that collapse in JEPA training isn't a regularizer problem but a supply–demand mismatch: the encoder H collapses first, so no gradient from Z reaches it.

Key claims:

The author notes LeJEPA is right about Z but misses the encoder-side collapse.

Original post →

More from Research

Research channel →