Under-the-radar Meta paper: JEM SSL framework beats DINOv3 at 7B with 12x less data
iScienceLuvr · x · 2026-10-26
Meta researchers' paper "Scalable Patch-Level Self-Supervised Learning" introduces JEM, a student-teacher SSL method derived from an information-theoretic multi-view objective: align patch representations across views, with explicit information- and structure-preservation regularization.
Key points:
- Trains stably from 300M to 7B parameters; reportedly the first latent-space patch-level method shown at 7B scale
- Strong performance on global and dense probing at all scales, consistently surpassing DINOv2 on segmentation benchmarks
- At 7B it exceeds DINOv3 on panoptic segmentation despite using 12x less data and no refinement stages
- Takeaway: a principled SSL algorithm can match ad hoc multi-objective stacks
Related event: Meta's JEM Self-Supervised Framework Beats DINOv3 with Far Less Data(2 posts)→
More from Research
- One week: 80-year-old cells reset to under 20, AI designing working proteins — rand_longevity · 2026-10-26
- Anthropic interpretability demo shows neurons combining into tiny pirate/wizard personas — Sauers_ · 2026-10-26
- Steering 'emotion neurons' in LLMs changes their social decisions — burny_tech · 2026-10-26
- 17-page HMM chapter PDF from Jurafsky & Martin's NLP textbook shared — burny_tech · 2026-10-26
- Two EMNLP mech-interp papers: LLM agents as circuit explainers and DuoSteer for safe code generation — ZiyuYao · 2026-10-26
- Lafa paper scales flow models to 8B by adapting AR weights, 2.54x AR throughput — burny_tech · 2026-10-26