New paper claims decoder LLMs are injective and can reconstruct exact input text
s_scardapane · x · 2026-07-29
Language Models are Injective and Hence Invertible argues that decoder transformer language models map discrete input sequences to continuous hidden representations in an injective way, so the input can be recovered exactly from activations.
- The authors provide a mathematical proof that this property holds at initialization and is preserved during training.
- They validate the claim with billions of collision tests across six state-of-the-art language models and report no collisions.
- They also introduce SipIt, an algorithm that provably reconstructs the exact input text from hidden activations in linear time.
- The paper frames injectivity as relevant to transparency, interpretability, and safe deployment.
More from Research
- Late-2010s generative AI recap highlights BigGAN, StyleGAN and old notebooks — Merzmensch · 2026-07-29
- ACL study finds non-English Wikipedia has major quality and bot-content issues — mdlhx · 2026-07-29
- New study says human TSP solutions are near-optimal but still systematically human — xuanalogue · 2026-07-29
- Google research on 15 million Gemini interactions finds little evidence of mass job automation — ChuckDBrooks · 2026-07-29
- Tencent open-sources AngelSpec, a speculative decoding framework with up to 2.4x speedup — teortaxesTex · 2026-07-29
- ICLR 2027 deadline falls 8 days before NeurIPS 2026 decisions — 1414vo · 2026-07-29