Frontier LLMs all lost money in a 1.6-year synthetic trading test
Scobleizer · x · 2026-07-22
Frontier LLMs all lost money on a synthetic trading benchmark
A SynthFin Trading Bench test put Fable, GPT-5.5, Gemini 3.5 Pro, and Grok 4.5 into portfolios on a fully synthetic stock market they had never seen before.
- The run lasted about 1.6 years across 1,000 stocks and 10 macro regimes.
- Every model lost money.
- None of the LLMs beat a simple buy-and-hold baseline.
- The chart shows equal-weight doing best at +4.84%, while buy & hold was roughly flat at +0.25%; the AI portfolios all finished negative, with Gemini 3.5 Flash the worst at -24.31% and Momentum at -26.53%.
The image also notes the benchmark was designed to be contamination-free: all prices, filings, and news were machine-generated and unpublished, so the models could not have memorized them.
More from Models
- Unverified: GPT-6.1 Terra reportedly released, leaked chart shows top score per dollar on Agents' Last Exam — ZeroStateReflex · 2026-10-27
- GPT-6.1 Terra cuts factual error rate from 22.3% to 4.3%, an ~81% drop — haider1 · 2026-10-27
- GPT-6.1 Terra lands: early users praise speed, value and subagent performance — BenBajarin · 2026-10-27
- Leaked GPT-6.1 Terra benchmarks: cheap but 'boring', still far behind Astra and Claude — Angaisb_ · 2026-10-27
- Artificial Analysis Ships Intelligence Index v4.3.2 With GPT-6.1 Terra Comparison — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra cuts hallucination rate by 50 points while lifting accuracy 2 points — ArtificialAnlys · 2026-10-27