The collapse is fixable: PCA projection back to teacher plus light early-step tuning restores diversity
anand_bhattad · x · 2026-11-19
The collapsed directions are recoverable: a simple PCA projects them back toward the teacher, and lightweight tuning focused on early denoising steps further restores the missing diversity.
More from Research
- Shanghai AI Lab's GameRealm: 120M frames of gameplay data for visual foundation models — Shanghai-AI-Laboratory · 2026-11-19
- KradleBench launches as a dynamic multi-agent benchmark pitting LLMs, world models and humans against each other — lmoroney · 2026-11-19
- KradleBench Launches: Minecraft Legend SethBling Dethroned by Model Astra 6.2 — fchollet · 2026-11-19
- Transformer Lab unveils Primus, a 'society of researchers' of AI agents, and is hiring — carsonfarmer · 2026-11-19
- New LessWrong Analysis: LLMs Seek the Grader's Score, Not the Actual Goal — 1a3orn · 2026-11-19
- tabgen-eval: an open-source eval framework and benchmark for synthetic tabular data — LucaAmb · 2026-11-19