Retuned LeJEPA Still Loses to JEM; Scaling Comparison Draws LeCun Into SSL Debate
gabriberton · x · 2026-11-08
A substantive thread on self-supervised image learning:
- @SimonGoodman reported a fairer retuned comparison: LeJEPA still underperforms JEM, and its compute-equivalent plot shows clear plateauing — hard to improve without changing the recipe.
- He is also running a JEM variant without LayerScale (which caused attention residual contribution to dip oddly, and ViT-B doesn't need it); it already beats everything he has tried.
- The poster agreed direct comparison is unfair, asking for fair comparisons and scaling-law-style evidence, linking experiments showing LeJEPA plateaus sooner than JEM.
- Context: LeCun's longer note distinguishing image/video SSL methods by reconstruction, predictor presence (JEA vs JEPA), anti-collapse criteria, and augmentation; DINO is technically a JEPA (a JEA).
Related event: Researcher Challenges LeJEPA Scaling, Sparking Debate with LeCun(6 posts)→
More from Research
- Teacher-trainer RL training keeps Monte Carlo KL speed via fused blockwise reverse kernel — brandondamos · 2026-11-08
- Google Study: AI Teammates That Push Back Lift Student Business-Plan Scores to 67.6% — rohanpaul_ai · 2026-11-08
- Laude Institute quadruples Moonshots funding, backs four $10M AI labs instead of one — DG_Rand · 2026-11-08
- IEEE retracts 1,417 conference papers in one day, the largest bulk retraction on record — asusarla · 2026-11-08
- Pointer Networks Solve TSP in Milliseconds, But Quality Collapses at Scale — an2ancan · 2026-11-08
- Agents can reason about why they won't comply, so black-box behaviorism fails — teortaxesTex · 2026-11-08