Investor's Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future?
Huangfei Jiang, Anning Hu, Kai Sheng, Yuchen Cai, Rui Ge, Xiangyu Kong, Jianghao Lin, Shuzi Chen, Jiyi Weng, Junchi Yan
cs.AI
2026-10-28
ILE, a buy-side-built equity research benchmark with a 3.43M-version frozen database, finds GPT-6 Astra leads at 68.1 overall but only 52.5 on Forecast & Model.
Financial agent benchmarks have a mechanical hole: look-ahead leakage. Information that did not exist at decision time reaches the model through two routes. Pretraining may have memorized outcomes, and retrieval-side date filters are unreliable. The paper opens with a concrete case: a GPT-6 Astra web query explicitly tagged before:2026-03-24 still returned a July 2 video page carrying Tesla's realized Q2 delivery numbers. A page's publication date need not reflect when its content became available, so updates and retrospective reports slip straight through date filtering.
Prior benchmarks each cover one slice. FinanceBench tests document QA, Finance Agent and FinSearchComp test search workflows, and FinanceGym added publication-date filtering, but that stays a retrieval-level heuristic governing neither computation nor scoring. ILE (Investor's Last Exam) turns the information boundary into a system-level constraint and swaps generic criteria for professional buy-side standards.
Three components.
Protocol: 11 systems, one shared tool library of six families, 7,200 seconds and up to three submissions per task. Numeric criteria are scored by deterministic checks, semantic criteria by GPT-6 Astra.
| System | Overall | Standout and weak spots |
| GPT-6 Astra | 68.1 | 78.9 on Research & Evidence and 77.6 on Evaluate & Decide, but 52.5 on Forecast & Model |
| Claude Opus 5.5 | 66.9 | Leads Forecast & Model (52.6) and Monitor & Update (64.2) |
| DeepSeek V4.1 Flash | 63.0 | $0.36 per task; Opus costs about 22x more for 3.9 more points |
| Ling 3.0 Flash | 42.6 | Bottom of the field |
The paper's hardest finding is that three orderings disagree.
Forecasting is a collective weakness: Astra's Forecast & Model score sits 26 points below its research score, with KPI and cross-company forecasting at 41.1 and 49.6. Under forecast-outcome criteria the ranking flips: DeepSeek V4.1 Flash leads at 40.3, ahead of Astra (37.7) and Opus (37.6), and DeepSeek V4 Pro wins Market Trading at 64.6 despite an overall score of 59.3. Simulated trading disagrees again. Signals from 20 assignments became long/short portfolios (March-August 2026, 10 bps per leg plus 3% annualized short borrowing): Hunyuan 4 Preview tops the table at +8.50%, Astra makes +7.18% against +1.18% for always-long, and Kimi K3 earns 7.15% versus Opus's 6.39% despite a lower research score.
One comparison explains why the bundled database matters. Astra scores 68.1 on the versioned corpus and 68.3 on public-web search, nearly identical, yet 57% of its public-web trajectories (49/86) touched post-cutoff sources. Same score, completely different information boundary.
Failures concentrate upstream of retrieval and arithmetic. On the AEP funding-gap task, all 11 systems recovered the eight reported inputs and reconciled both cash-flow identities, but only 5 computed both funding shortfalls, 3 separated net from gross fuel adjustments, and 5 reached the supported conclusion. Submission reliability is a separate failure source: under strict scoring, Step 5 Preview drops from 55.1 to 34.5 and Hunyuan from 60.4 to 47.7, with Hunyuan leaving 17 trajectories unsubmitted.
For teams building financial agents, the leakage-proof architecture is the transferable part: versioned corpora plus toolchain-level authorization checks beat prompting a model not to peek, and the design can be copied into internal evals. Diagnostic granularity is the second value; dependency graphs locate errors at specific steps. The task suite and code are public, with rubrics kept private.
For model selection, the aggregate score is close to meaningless here. Research quality, forecast accuracy, and simulated returns produce three inconsistent orderings, so which model to pick depends on which capability the downstream task actually needs.
The authors flag two. The versioned corpus only blocks retrieval-side leakage; frontier models may have memorized historical outcomes during pretraining, and separating parametric memory from retrieval will require counterfactual or synthetic filings. Also, some tasks settle against realized outcomes, so market noise can push a sound analysis off the realized numbers, and outcome-only scoring would underestimate research quality.
Open questions from the reading: 86 tasks and 20 simulated trades is a thin base for claims like ranking reversals; the gap between +8.50% and +7.18% over 20 trades may well be noise, and no significance test is reported. GPT-6 Astra grades the semantic criteria while also competing, so self-enhancement bias is mitigated by a sensitivity appendix but not eliminated. The benchmark is also designed to expand with new disclosures, which means scores taken at different times are not directly comparable.