Gemini's Stock Predictions: More Confident, More Wrong
aufgeblobt · reddit · 2026-07-15
The author created a continuous, time-locked stock prediction dataset to evaluate **Gemini 2.5 Flash** without look-ahead bias. Using Google Search grounding, the model predicted stock price directions, sentiment, confidence, and full reasoning chains for the next 10 trading days. The experiment ran from 2026-02-17 to 2026-05-19, exceeding 90 days. Key finding: **the more confident the model, the more it erred**. The overall Expected Calibration Error (ECE) was 0.217, with accuracy dropping to 28% in the 0.8–0.9 confidence range. A "downside risk" prompt aimed at correcting LLM over-optimism led to overcorrection, causing 'Down' predictions to exceed actual market downturns. Additionally, a "phantom pivot" detector—designed to catch hallucinated trend reversals—plateaued at roughly 50% on high-volume tickers. The author has open-sourced the methodology, dashboard, and results online, with a multi-model dataset available on HF.
More from Research
- A Matrix meme turns an LLM-solved-problems debate into a question of belief and access — prasanna_says · 2026-07-21
- More compute can materially improve frontier models’ cyber benchmark performance — peterwildeford · 2026-07-21
- Paper claims stochastic exploration fixes two 3D Gaussian Splatting optimization bottlenecks — zhenjun_zhao · 2026-07-21
- SSR refines monocular geometry with sparse volumetric updates and sparse 3D U-Nets — zhenjun_zhao · 2026-07-21
- Survey of 300+ papers says better reasoning does not make LLMs more self-aware — blaizedsouza · 2026-07-21
- A research note argues identity preservation should be a measurable requirement — maier_ak · 2026-07-21