LeCun paper: 109M-parameter model beats 72B models on representation matching, 660x smaller
examachine · x · 2026-11-02
A paper by Randall Balestriero and Yann LeCun examines why models trained on text and images develop similar internal representations. A 109M-parameter model beat models up to 72B — roughly 660x larger — on how closely its representations matched image models, showing scale alone doesn't explain it. Training pairs matter: two observations of the same scene give a shared concept to learn, while next-word prediction provides a weaker signal, with the better-paired method needing 42x fewer examples.
Related event: LeCun's New Paper: 109M-Parameter Model Beats 72B LLM in Representation(4 posts)→
More from Models
- Is Claude Opus 5.5 alright? User flags sloppy behavior over unknown tools and API names — JeremyNguyenPhD · 2026-11-02
- SemiAnalysis releases GDPx agentic benchmark; best model scores just 12%, Pichai nods — sundarpichai · 2026-11-02
- Insider claims astra outsmarts Opus and Fable but was benchmark-optimized at the cost of alignment — haider1 · 2026-11-02
- JetBrains ships Mellum2.1 Thinking MTP head: 1.4–1.8x faster decoding with ~90% draft acceptance — pmttyji · 2026-11-02
- Grok Bot pricing debate: users say free tier makes no sense, back gifting instead — RachelVT42 · 2026-11-02
- Grok users hit weekly limits early as OpenAI doubles Pro caps — how do teams plan? — Entire_Struggle_2236 · 2026-11-02