Debate: Models Fuzzily Recall Concepts, Not Text — SAE Features vs Edit-Distance Memorization
voooooogel · x · 2026-09-05
A technical debate on LLM memorization and SAE interpretability. The author is skeptical that models verbatim-memorize training data in ways measurable by string edit distance — what he's seen is more like fuzzy recall of concepts.
Example: a model likely won't remember the exact text of a specific message-board post, but could hypothetically carry an SAE feature like 'writing message-board entries starting with multiple Z as an obfuscation mechanism' — abstract patterns rather than exact strings. The thread also touches on model size and scaling behavior in RL.
Related event: Debate Erupts Over SAE Features and How Models Remember(2 posts)→
More from Models
- Tencent's WeChat Vision releases WeVisOne-VL, unifying object- and pixel-level perception — jacek2023 · 2026-10-23
- 105 Real Bugs Tested: GPT-6.1 Sol Over 10x Cheaper Than GPT-5.6 Sol — PawelHuryn · 2026-10-23
- Abliterated V4.1-Flash eval: abstract algebra drops 6pp but college math gains 5pp — teortaxesTex · 2026-10-23
- Claude users hit "model at capacity" errors hourly, canceling plans over it — Alex-Stack · 2026-10-23
- Kimi K3.1 launch imminent: 3T-param bet on RL seen as make-or-break — teortaxesTex · 2026-10-23
- Uncensored fine-tunes vanish from Hugging Face amid unexplained takedowns — umarmnaq · 2026-10-23