teortaxesTex asks if append-only reasoning over long agentic sequences is just continual learning
teortaxesTex · x · 2026-11-03
teortaxesTex 在评论新注意力方法时提出：既然现在可以在极长的 agentic 序列上做追加式（append-only）推理，这与其被反复承诺的「持续学习」有何本质区别？能否让模型一直推理下去？他认为即便有检索，残差流容量限制与上下文表征相对训练的漂移仍会带来认知过载，因此离终局尚远。被引用的 Beren Millidge 原推则称其方法采用了正确的树形数据结构，注意力本质上是稀疏且分层的。
More from AGI Musings
- Software is shifting from winner-take-all to a restaurant-style long-tail market — tinyfool · 2026-11-03
- The Russell Conjugations of Intelligence: Why 'Just Matrix Multiplication' Offends Us for Brains but Not for AI — esrtweet · 2026-11-03
- AI safety researcher: assuming all emergence is risk may be the wrong frame — repligate · 2026-11-03
- AI bubble burst favors pure ML backgrounds: end-to-end skills now beat niche expertise — kmeanskaran · 2026-11-03
- Red teamer: jailbreak work isn't ethically neutral and takes a toll on humans and models — repligate · 2026-11-03
- repligate: "AI welfare" was fringe a year ago, and its framing is already a concession — repligate · 2026-11-03