MDLM-like Models Flaw: Independent Conditionals Can't Represent Joint Distribution
kalomaze · x · 2026-08-10
kalomaze argues that MDLM-like models learn independent conditionals, not a joint distribution. For instance, 'New Carolina' isn't a real state, but parallel samples of decoupled token conditionals are blind to this. Tricks can make the independence assumption less incoherent, but the generative modeling floor remains.
Related event: Developer Highlights Theoretical Blind Spot in Discrete Diffusion Models(3 posts)→
More from Research
- MOSS-Transcribe-Diarize: Transcription and Diarization in One 0.9B Model — vanstriendaniel · 2026-08-10
- Harvard and MIT Open-Source MatrAIx: Simulating 8.3B Global Personas — SRSchmidgall · 2026-08-10
- DAP: Open-Sourced Foundation Model for Panoramic Depth Estimation — tom_doerr · 2026-08-10
- SFT Conflicts, RL Coexists: Theoretical Analysis of Multi-Task LLM Training — CASIA · 2026-08-10
- Zero Gap Is Not Restoration: SA-PPG Metric and RailCap for Benchmark Contamination — zju · 2026-08-10
- Beyond Environment Scaling: Effective Distributions for Multimodal Agent Learning — CASIA · 2026-08-10