Directly supervising representation dot products with generative-model PMI estimates hailed as a new paradigm
vincesitzmann · x · 2026-10-20
Vince Sitzmann championed Alan Baade's representation learning work, calling the idea of directly supervising the dot product of representations with a PMI estimate from a generative model brilliant and non-obvious — potentially a new paradigm in the field.
The approach also establishes a clear link between representation learning and generative modeling: generative models are arguably the best tools ever built for measuring information and mutual information, yet they don't consolidate that information into a compact representation. This work bridges that gap.
More from Research
- mjfit: open-source convex decomposition for tight-clearance parts, 10x faster than SDF — yacineMTB · 2026-10-20
- iSEE world model learns object permanence from video alone, no labels — CSProfKGD · 2026-10-20
- TasteBench tests model research taste on ever-fresh out-of-sample ML tasks — marktenenholtz · 2026-10-20
- "Neurons aren't already clean — interpretability just covers the dunghill with snow" — 1a3orn · 2026-10-20
- Single neuron steering makes Qwen3-8B insist it's an owl — Sauers_ · 2026-10-20
- Q-CTRL: over 99.5% of qubits sit idle in one fault-tolerant Shor's algorithm blueprint — MJBiercuk · 2026-10-20