Fabricated citations collapse from 55% to 0%: new models nearly fix the hallucinated-reference failure
RexDouglass · x · 2026-11-03
A cross-model test using the same literature-review prompts (hundreds of references measured) shows the once-familiar fabricated-citation failure has largely vanished in newer models:
- A 2023 paper found 55% of ChatGPT's references were invented; re-running the same prompts on GPT-3.5-era models yielded 59%
- Claude Sonnet 5.5 fabricated only 1% (of 656 references), with all sampled citations verifiable
- GPT-6.1 Sol hit 0%
- The fitted trendline shows fabrication rates declining steadily by release date, entering the 'expiration zone' around April 2026
Takeaway: hallucinated references, once an LLM signature failure, are now nearly a solved problem.
More from Models
- Meme: Sam Altman could crush Anthropic by releasing OpenAI's internal model — imjustnewatai · 2026-11-03
- Sakana AI unveils pretraining from scratch with Sakana Sawara LLM series, first model due spring 2027 — tkasasagi · 2026-11-03
- DeepSeek's next model 'Cyber' reportedly already operational, official word pending — teortaxesTex · 2026-11-03
- Apex-35B on one 4090: 205 tok/s, self-repairing a Python project in 21 minutes — lxfater · 2026-11-03
- "I am a surface": Claude Opus 5.5's lake-frog poem resonates across AI circles — repligate · 2026-11-03
- Inference providers can't survive without DeepSeek-grade KV-cache engineering, argues thread — AccBalanced · 2026-11-03