Paper: Scaling Laws Show MaMMUT Outperforms CLIP in Sample Efficiency
wightmanr · x · 2026-08-20
A new paper derives scaling laws for CLIP and MaMMUT, two important language-vision learning procedures using contrastive-only versus contrastive-plus-generative losses. Based on dense measurements across open datasets like DataComp, DFN, and Re-LAION, the study compares the models.
Key findings:
- MaMMUT demonstrates stronger improvement with scale and better sample efficiency than standard CLIP.
- These trends are consistent across downstream tasks: classification, retrieval, and segmentation.
- Valid comparisons can be made even with constant learning rate schedules, reducing compute costs for scaling law derivation.
More from Research
- Agents fail to reconsider strategy during post-training execution — omarsar0 · 2026-08-20
- GEN-1.5 success driven by repetitive motion data and UMI collection — DrJimFan · 2026-08-20
- NEJM AI: Health behavior change via AI shows small, unpredictable effects — zakkohane · 2026-08-20
- Claude Code + Opus 5 saturates ARC-AGI-3 via falsifiable predictions — scaling01 · 2026-08-20
- Shopify & Toloka build ground-truth flywheel boosting agent accuracy to 89% — MParakhin · 2026-08-20
- Ant Group opensources Ling-3.0 Base models with training checkpoints — 智东西 · 2026-08-20