Two studies: AI chatbots get financial advice wrong 57%-85% of the time
PtrPomorski · x · 2026-09-23
Two studies on two continents reach the same conclusion: general-purpose AI chatbots get financial advice wrong far more often than right.
- Saturn (UK): 57% overall failure rate, 88% on complex questions
- DeepVest (US): 85% failure rate on portfolio calculations
- Even calculation-free "easy" questions: only 54% accuracy
- Inconsistency: leading chatbots gave meaningfully different answers to identical questions at different times
- CEO Toby J. Wade: "General-purpose AI models are trained to sound confident, not to be correct"
More from Models
- Claude Opus 5.5 Debuts: 40% Cheaper, ~30% Faster Than Opus 5, Stronger Agentic Coding — nicolechirps · 2026-09-23
- Fireworks Launches Specialized Intelligence Index; DFS Model Hits 62.2% Vuln Detection Recall — nicolechirps · 2026-09-23
- Sol 6 matches Sol 5.6 quality with half the reasoning time, better Plus limits — Simple-Diver-2192 · 2026-09-23
- Founder: OpenAI should ship its stronger internal model and cut Astra prices 50% — bindureddy · 2026-09-23
- No Single Champion: Model Rankings Shift by Domain, from Legal to Finance to Healthcare — sophiamyang · 2026-09-23
- Quintin Pope asks if Claude shows in-context emergent misalignment — QuintinPope5 · 2026-09-23