giffmana questions LeJEPA: no scaling law evidence vs DINO
giffmana · x · 2026-11-07
In a discussion on LeJEPA's comparisons, DeepMind researcher Luca Eyre (giffmana) notes the public weights were only trained on IN1k/IN21k — far from DINO v2's 140M curated images or DINO v3/SigLIP's billions — and there's no evidence the method scales to large datasets. A proper scaling-law plot of LeJEPA vs DINO would be far more convincing; single-point scores are nearly meaningless.
Related event: Researchers Question LeJEPA's Scalability to Larger Datasets(2 posts)→
More from Research
- Turn gaps average ~200ms: duplex voice models must plan replies mid-listen — smolix · 2026-11-07
- Why Higgs Realtime Duplex uses a 160ms step: it matches human decision speed — smolix · 2026-11-07
- ByteDance experiments suggest Full Bandwidth Transformer matches models twice its size — DimitrisPapail · 2026-11-07
- Ai2 brings 10+ robotics papers to CoRL 2026, spotlighting MolmoAct2 and MolmoBOT — allen_ai · 2026-11-07
- Applied Compute shares a production RL recipe for continual learning from live traffic — ypatil125 · 2026-11-07
- Anthropic Science Blog: AI Begins to Write, Fit and Revise Scientific Models — AnthropicAI · 2026-11-07