Could Writable Ngram Embedding Tables Enable Inference-Time Learning? A Reddit Proposal
RapidRaid · reddit · 2026-09-12
A Reddit poster proposes adding a writable "live" table on top of pretrained read-only ngram embeddings (as seen in models like Qwen and DeepSeek): new knowledge writes new embeddings, frequent entries persist, unused ones decay — mimicking memory formation and forgetting during inference. Open discussion, no implementation yet.
More from Research
- 100 LLM Agents Run a Town Economy for 26 Weeks — and Money Stops Moving — omarsar0 · 2026-09-12
- kalomaze: You don't need an analytic transfer theory, just a learnable transfer-extrapolation function — kalomaze · 2026-09-12
- Simons Institute holds workshop on AI's rapid acceleration of mathematics and theoretical CS — jasondeanlee · 2026-09-12
- Dev Fine-Tuned a 2B LLM on WhatsApp Group Chat, Simulating Six Friends on an M1 Pro — BarisSayit · 2026-09-12
- First quantitative evidence: Claude and GPT now use GUIs as well as APIs — ysu_nlp · 2026-09-12
- World Models Will Power the Next Leap in AI Agents — And They May Never Show Video — furongh · 2026-09-12