antirez Clarifies LLM Hidden States Are Not Full Internal Representations

Redis creator antirez clarified that LLM hidden states do not carry a complete internal representation of the model, noting that the last token's hidden state only holds what is needed to predict the next token, making similarity-based retrieval inherently limited.

2026-11-04 ~ 2026-11-04 · 3 related posts

• antirez: An LLM's Hidden State Isn't Its Full Representation — It's Not an Encoder — antirez · 2026-11-04
• Antirez argues LLM hidden states aren't encoders — they only carry what the next token needs — zafstojano · 2026-11-04
• antirez: LLM last-position hidden states have inherent limits for similarity — antirez · 2026-11-04