Quant-Arena benchmark: 5 of 9 LLM quant strategies lose money, none beat 17.1% baseline
jiqizhixin · x · 2026-11-02
Navers Lab's Quant-Arena benchmarks the full quantitative-research loop rather than isolated trading predictions. Agents write stock-scoring code for 3,042 A-share stocks using daily and Level-2 data, may evaluate up to five candidate strategies on 2022–2023 data, then freeze one for a single assessment on sealed 2024 data. A shared evaluator accounts for execution constraints, transaction costs, and market impact, and assessment results never flow back to the research agent.
Key findings:
- 5 of 9 representative model strategies lost money in the sealed 2024 assessment
- None beat the 17.1% gross equal-weight market benchmark
- Research feedback improves composite scores more consistently than out-of-sample returns, exposing a gap between research-stage self-improvement and real profitability
The diagram details the two-phase pipeline: research (hypothesize → build → evaluate, ≤5 evaluations, ≤300 declared features) and sealed assessment (frozen code, Top-75 equal-weight long portfolio, final score S = 0.35·Sin + 0.7·Sout).
More from Models
- Phil Chen: the biggest unsolved problem in LLM autoresearch is bypassing Anthropic safeguards — philhchen · 2026-11-02
- Opus 5.5 is good in exactly the ways DeepSeek is good, but not a deep thinker — teortaxesTex · 2026-11-02
- DeepSeek V4.1 Flash 1017 Scores 159 on Epoch ECI, Tied With GPT 5.5 — teortaxesTex · 2026-11-02
- Domingos: If OpenAI and Anthropic Had a Secret Sauce, Distillation Wouldn't Work — pmddomingos · 2026-11-02
- AI commentator laments frontier labs leak nothing, not even model configs — teortaxesTex · 2026-11-02
- First GPT-6.1 Terra test: perfectly rebuilds a scanned PDF form in 81 seconds — MikePFrank · 2026-11-02