Meta's overlooked JEM SSL framework beats DINOv3 with 12x less data
iScienceLuvr · x · 2026-10-26
Meta FAIR researchers published an under-the-radar paper, "Scalable Patch-Level Self-Supervised Learning," introducing JEM — a principled student-teacher SSL framework that aligns patch representations across views, with explicit regularization (an InfoMax term preventing collapse and a structure-preservation loss). JEM trains stably from 300M to 7B parameters — reportedly the first latent-space patch-level method shown at 7B scale. It consistently beats DINOv2 on dense tasks like semantic and panoptic segmentation, and at 7B it surpasses DINOv3's panoptic segmentation despite training on 12x less data with no refinement stage, while staying competitive on image-level classification.
Related event: Meta's JEM Self-Supervised Framework Beats DINOv3 with Far Less Data(2 posts)→
More from Research
- Phantom Transfer: data poisoning survives all 11 tested data-level defences — sebkrier · 2026-10-26
- Titans learns during inference—but what if it learns the wrong thing? An experiment shows recall never fully recovers — pardhu-- · 2026-10-26
- Hugging Face publishes a five-step curriculum for post-training AI agents — lmoroney · 2026-10-26
- 125 poisoned examples planted a code-stealing backdoor in Qwen 2.5-7B — GarrisonLovely · 2026-10-26
- HAPAX paper: abstractive ICL emerges even without induction-head copying — davidbau · 2026-10-26
- HiP3R stitches local 3D reconstructions to keep long-video scenes consistent across thousands of frames — ccloy · 2026-10-26