repligate argues context full of noise is genuinely 'hard' for LLMs, mulls an info-theory model of breaks
repligate · x · 2026-10-27
In a thread with @EpistemicHope, AI researcher repligate (70K followers) discusses why breaks help humans and whether transformers have analogous mechanisms:
- repligate argues the benefit of breaks may be not just neurobiological but "information theoretical" — pointing to an ambitious, possibly not-yet-existing version of information theory.
- His concrete claim: when a context fills up with noise and stuff to track, it is legitimately "hard" for LLMs.
- The counterpart pushes back: known mechanisms live at the level of synapses and neurotransmitters — what's the info-theory model of how breaks help?
A substantive in-circle debate on LLM behavior and human cognition analogies worth following.
Related event: Cluttered context is genuine difficulty for LLMs, researcher argues(2 posts)→
More from Models
- Codex ban update: users jailbreak AI agents to bypass Daybreak filter for reverse engineering — FlolightC · 2026-10-27
- GPT-6.1 Terra hits GA in GitHub Copilot, stronger terminal coding at lower cost than GPT-6.1 Sol — 0xkarasy · 2026-10-27
- PostTrainBench creator notes Muse Spark 1.3 system card files it under loss-of-control risk — maksym_andr · 2026-10-27
- Mathematicians debate AI math writing: abysmal today, improving shockingly fast — miniapeur · 2026-10-27
- LessWrong: GLM models show outlier chain-of-thought controllability vs Kimi and DeepSeek — tomekkorbak · 2026-10-27
- Moss image model 'ghost duplicates': single-person prompts spawn phantom copies — No_Drag8242 · 2026-10-27