Model got numbers right but sources wrong 3/3 times
Virtual-Flow-2784 · reddit · 2026-09-02
A user observed that a model correctly selected financial numbers 3 out of 3 times, but hallucinated the source for the data every time. The author notes that while the model looks perfect if scored only on numerical accuracy, real-world finance workflows require verifiable provenance. This suggests that accuracy and provenance should be scored as separate metrics.
More from Models
- Fable and Mythos 5.1 share exact same weights, differing only in classifier fallback — zsakib_ · 2026-09-02
- Benchmark Scores Don't Matter; Open Local Models Are the Future — johnseach · 2026-09-02
- OpenAI & Anthropic Dominate; Rivals Lag Behind — emollick · 2026-09-02
- OpenAI's unreleased Astra model found V8 zero-days during testing — kimmonismus · 2026-09-02
- OpenAI's Astra model finds and chains V8 zero-days to gain root access — kimmonismus · 2026-09-02
- Comparison: Fable 5.1 vs GPT Astra Shows Significant Token Usage Gap — Linahuaa · 2026-09-02