New proof: perfect reconstruction can't guarantee VAE latent codes read back as written
Hidenori8Tanaka · x · 2026-10-20
Yusuke Hayashi's new paper shows a hidden assumption in systems that read VAE latents as discrete codes — that code i comes back as i — is unenforced and unverifiable from standard training reports:
- Impossibility: no item of a standard training report reveals whether written and read codes agree on encoded data
- Constructive counterexamples: two VAEs with perfect reconstruction, exact inference, and identical per-example ELBO and code histograms — one with agreement 1, the other 0
- Bounds: the ELBO's variational gap bounds shifts to exact inference; the rate bounds shifts to prior generation; mean-field Gaussian encoders pay an exact price depending on axis choice
- Experiments: on probabilistic PCA the gap certifies encoders in all 75 settings (agreement 0.76–0.97); across 240 trained VAEs, 66 are ruled out in advance and none certifies
The paper proposes a Certify-or-Measure protocol for any pipeline reading VAEs as codes.
More from Research
- Spherical Fixed-Point Reasoners: keeping looped transformers bounded and teaching them when to stop — GabGarrett · 2026-10-21
- First ever: AI-designed bacteriophages dosed to a human patient — JosephJacks_ · 2026-10-21
- Perplexity ships Decider V1.2, tops Vals AI's new ClaimBench decision-model benchmark — bo_wangbo · 2026-10-21
- Masking giveaway terms barely dents LitGene's gene-relation scores, study shows — bravo_abad · 2026-10-21
- Omnii-1 genome model debuts: first AI-designed phages ever given to a human patient — JosephJacks_ · 2026-10-21
- How LLMs store facts: three learning phases, attention circuits, and hallucinations — scott_linderman · 2026-10-21