LeCun proposes a generalization theory framework for self-supervised learning cost-regret tradeoffs
ylecun · x · 2026-10-11
Responding to Andrew Ng's question about open problems in understanding generalization, LeCun sketches a generalization theory for self-supervised learning: assume unlabeled samples cost X each and task labels cost Y each; analyze the regret curve (task error vs. number of labels) when X is zero, and for K tasks, find the optimal numbers of unlabeled samples n and per-task labels m minimizing total cost nX+mKY to hit a target error rate.
More from Research
- OpenTac auto-routes tactile sensor PCBs from a 3D model in 3 seconds, covering 93% of surface — anand_bhattad · 2026-10-23
- arXiv paper gives StopGrads theoretical grounding: convergence without an underlying loss — srush_nlp · 2026-10-23
- PEARL demo: an LLM agent that checks experiment results, revises plans, and re-tests — KevinKaichuang · 2026-10-23
- Berkeley's StyleStream 2.0 cuts streaming voice conversion latency to 520ms, 2x faster — Berkeley · 2026-10-23
- Math PhD student quits academia: AI is making math unrecognizable — pickover · 2026-10-23
- ICLR Submission Found Containing LLM's Own Meta-Comment Admitting Results Weren't Significant — _onionesque · 2026-10-23