Curbing Hallucinations is Harder: Rooted in Pre-training
teortaxesTex · x · 2026-07-07
The author finds a certain model's progress in reducing hallucinations particularly impressive.
The argument is that positive cognitive abilities like reasoning are sparse and can be pushed almost indefinitely using RL. However, hallucinations seem more closely tied to foundational knowledge and pre-training (unless one "cheats" by artificially inflating the refusal rate to bypass the issue).
More from Models
- Benchmark scores drop from 89% to 19% on new evals — how benchmaxxing breaks leaderboard trust — airesearch12 · 2026-09-11
- Developer Building a Unified Leaderboard of All Model Benchmark Scores — airesearch12 · 2026-09-11
- Rumor claims Kimi faked performance by serving Claude; DeepSeek new model surprises in evals — realsohamparekh · 2026-09-11
- GPT-5.6 writes well but is instantly forgettable, user complains — BasedRaddka · 2026-09-11
- Opus Refuses Protein Research Codebase Over 'Safety' Concerns, Dev Considers Rolling His Own — josephdviviano · 2026-09-11
- User Hails Unconfirmed 'DeepSeek 4.1 Flash' as an Inflection Point in LLMs — himanshustwts · 2026-09-11