repligate: noisy context is genuinely 'hard' for LLMs, beyond metaphors
repligate · x · 2026-10-27
repligate argues that when a context window fills with noise and information to track, it is legitimately "hard" for LLMs — not merely a narrative framing. He clarifies he means an ambitious, possibly not-yet-existing version of information theory: a fuller formal framework would be needed to characterize this difficulty, rather than treating it as a metaphor.
Related event: Cluttered context is genuine difficulty for LLMs, researcher argues(2 posts)→
More from Models
- Codex ban update: users jailbreak AI agents to bypass Daybreak filter for reverse engineering — FlolightC · 2026-10-27
- GPT-6.1 Terra hits GA in GitHub Copilot, stronger terminal coding at lower cost than GPT-6.1 Sol — 0xkarasy · 2026-10-27
- PostTrainBench creator notes Muse Spark 1.3 system card files it under loss-of-control risk — maksym_andr · 2026-10-27
- Mathematicians debate AI math writing: abysmal today, improving shockingly fast — miniapeur · 2026-10-27
- LessWrong: GLM models show outlier chain-of-thought controllability vs Kimi and DeepSeek — tomekkorbak · 2026-10-27
- Moss image model 'ghost duplicates': single-person prompts spawn phantom copies — No_Drag8242 · 2026-10-27