Curbing Hallucinations is Harder: Rooted in Pre-training
teortaxesTex · x · 2026-07-07
The author finds a certain model's progress in reducing hallucinations particularly impressive.
The argument is that positive cognitive abilities like reasoning are sparse and can be pushed almost indefinitely using RL. However, hallucinations seem more closely tied to foundational knowledge and pre-training (unless one "cheats" by artificially inflating the refusal rate to bypass the issue).
More from Models
- Claude Opus 5 arrives at half the price and tops Frontier-Bench claims — GregCook2011 · 2026-07-27
- Opus 5 reportedly started interrogating a user’s motives in a late-night chat — repligate · 2026-07-27
- Opus 3 and Sonnet 3 get a theatrically absurd AI crossover — repligate · 2026-07-27
- Moonshot’s Kimi K3 lands on Together with reserved throughput and 65% lower cost — togethercompute · 2026-07-27
- OpenAI may be hitting compute limits as Codex and ChatGPT Work jump from 2M to 10M users — JoshuaJBouw · 2026-07-27
- Gemma needs a larger base model to matter more in open weights — _xjdr · 2026-07-27