antirez: An LLM's Hidden State Isn't Its Full Representation — It's Not an Encoder
antirez · x · 2026-11-04
antirez offers a key clarification in the JEPA vs LLM debate around the platonic representation hypothesis: an LLM's hidden state does not carry the model's full internal representation—only what's needed to predict the next token. An LLM is not an encoder, so its hidden states can't be directly compared with JEPA's learned representation spaces.
Related event: antirez Clarifies LLM Hidden States Are Not Full Internal Representations(3 posts)→
More from Models
- Cohere Embed 5 hits OpenRouter: index with Pro, query with Fast in one shared embedding space — cohere · 2026-11-04
- RSI era but frontier models still take weeks: users question lab release pace — iruletheworldmo · 2026-11-04
- DeepSeek v4.1 flash provider shortage: Baseten slow, official API overloaded — shensi · 2026-11-04
- Editing 0.13% of weights, no training, fixes LightOnOCR-2-1B blank-page hallucination — vanstriendaniel · 2026-11-04
- antirez: benchmark games trivially guessable from output biases; Astra misses Cannon — antirez · 2026-11-04
- antirez replicates MIT weight-rewriting paper, finds visible radial patterns in first-layer weights — antirez · 2026-11-04