'Nothing new': researcher says engram overfitting is plain overfitting tied to over-parametrization
teortaxesTex · x · 2026-10-11
Responding to a claim that engram/n-gram approaches markedly amplify overfitting under data repetition, giffmana argues it's nothing special: it's plain regular overfitting that shows up at clear epoch boundaries, driven by over-parametrization rather than engram specifically. Engram just makes it cheaper to reach, as does MoE. He adds it isn't necessarily bad, depending on the plan.
Related event: Debate over engram overfitting under data repetition(2 posts)→
More from Models
- LLMs Reward Information Gain — Just Like the Best Humans Do — sanderssays · 2026-10-11
- Microsoft's decision model promised 80ms, serves 300ms via OpenRouter — DotaMate · 2026-10-11
- Qwen, Kimi and GLM dropped full attention — 8 attention designs explained — julsimon · 2026-10-11
- Meta's Muse growth slowing: daily active user gains down 62.4% from September surge — AccBalanced · 2026-10-11
- Researcher claims OpenAI exploits user prompts, cites Tao atop a list — basedjensen · 2026-10-11
- Leaked: OpenAI's unreleased model solved most math problems in a single prompt, ~3h each — Puzzleheaded-King584 · 2026-10-11