Apple researchers propose probe guidance, cutting guidance cost for diffusion LMs with no extra forward pass
itsbautistam · x · 2026-09-19
Apple MLR intern Rohit Dilip and colleagues published a preprint introducing probe guidance, a new way to guide flow matching models:
- How it works: builds a guidance signal from frozen internal states of an existing diffusion model, akin to autoguidance but without an extra inference-time forward pass, while keeping weak/strong model dynamics aligned.
- Results: sets a new SOTA for unconditional generation on continuous diffusion language models; on a 1.7B diffusion LM it consistently improves multiple-choice QA benchmarks.
- Insight: probe-based analysis shows autoguidance works only when the weak model comes from a low-entropy region of training, shedding light on its previously murky mechanism.
A near-zero-cost recipe for improving diffusion language models.
More from Models
- GPT-6 Astra builds a flamethrower demo in Three.js from a screen recording — techartist_ · 2026-09-19
- ChatGPT Pro 20x subscription returns after capacity-driven pause — mark_k · 2026-09-19
- Another FrontierMath open problem solved in an interactive GPT-6 Astra session — Jsevillamol · 2026-09-19
- Databricks: shifting just 20% of coding traffic to OSS models dramatically cuts inference spend — Yuchenj_UW · 2026-09-19
- Neuralink participant speaks in his own voice via brain implant powered by Grok — DimaZeniuk · 2026-09-19
- Devs say new Jev model is fast and exponentially cheaper for bulk tagging tasks — ivan_bezdomny · 2026-09-19