隐藏状态能否携带超出下一词的信息？研究者回应 antirez
burny_tech · x · 2026-11-06
Responding to antirez, researcher mlpowered addresses two questions: (1) Does a hidden state at token T encode more than what's needed for T+1? Unambiguously yes — the architecture allows it and there are concrete examples. (2) How often does it happen? That's an empirical question, and in practice it occurs frequently, not as an anomaly.
「研究」频道最新
- ICLR 审稿人质疑论文新颖性：工作坊版、arXiv 版谁是原创？ — BlancheMinerva · 2026-11-07
- 新论文:Optimistic Hedge 悔界取决于玩家数,三人博弈指数爆炸 — roydanroy · 2026-11-07
- Berkeley 等发布 Exo 架构：agent 边干活边改自身代码，省 35-61% 成本 — dair_ai · 2026-11-06
- 符号信念修正+几何质量投影，RAG重排序提升至0.481 nDCG@10 — letandrewcook · 2026-11-06
- 为 AI 智能体立宪:Charter 用 Lean4 打造形式化验证的规则语言 — QuixiAI · 2026-11-06
- ICLR 2027 收稿 42790 篇，57% 评审打 2 分，2-2-3 占 18.9% — CSProfKGD · 2026-11-06