Forecast Collapse in Time-Series Foundation Models
Shu Wan, Miles Ma, Hank Zhu, Guangqi Liu, Stephen Wang, Qingsong Wen, Huan Liu
cs.LG, cs.AI, cs.CE, stat.AP, stat.ML
2026-08-14
Time-series models trained with MSE produce nearly flat forecasts with poor stock ranking on US equity returns; the paper proves this follows from low predictability plus per-series objectives, and a combined MSE-plus-IC loss (CalibRank) nearly triples cross-sectional correlation across all twelve tested models.
Point a time-series foundation model (TSFM) at hourly returns for 1,000 US equities and something odd happens: the forecast is nearly flat, and its ability to rank stocks, measured by cross-sectional correlation (the quant industry's IC), hovers around 0.05. The paper names this forecast collapse. The puzzle sharpens on trading volume from the same stocks under the same protocol: collapse mostly disappears. Same models, same pipeline, different target, different fate. That rules out architecture as the cause. The paper shows collapse follows from two statistical mechanisms, which means a flat forecast can be the correctly calibrated response to a barely predictable target rather than a training failure.
Both mechanisms are formalized:
Together these define a calibration-ranking tradeoff: pure MSE is calibrated but flat; pure IC training ranks well but leaves scale free. The fix, CalibRank, is one line: squared error plus λ times a differentiable per-timestamp Pearson correlation term. The first term anchors scale, the second rewards ordering. Sweeping λ shows the first nonzero ranking weight captures nearly the entire IC gain; larger weights mostly inflate amplitude and error.
Experiments run on Finance1K, a panel the authors release: 28,510 hourly observations of 1,000 US equities from 2015 to early 2026, with aligned return and volume targets; first 19,957 steps train, the rest test. One protocol covers twelve forecasting backbones (DLinear, PatchTST, iTransformer, Autoformer, Non-stationary Transformer, and more). On the reference encoder:
| Objective | Amplitude (vs target) | Cross-sectional IC | MSE |
| MSE | 0.027 | 0.046 | 4.30×10⁻⁴ |
| IC only | 25.26 | 0.124 | 4.22×10⁻¹ |
| CalibRank | 1.836 | 0.126 | 1.43×10⁻³ |
IC rises 0.046 → 0.126 while amplitude stays within 2× of target; IC-only training reaches the same ordering at 25× target amplitude and two orders of magnitude more squared error. All twelve backbones improve, mean IC 0.062 → 0.123. A downstream check outside the loss, a long-short decision portfolio's held-out return, rises from 0.052 to 0.170. Zero-shot TimesFM and Chronos across 97 GIFT-Eval configurations show raw amplitude correlates with achieved predictability at 0.88 and 0.87: in the 65 configurations with R² ≥ 0.30 the models keep about three quarters of target amplitude; in the 6 with R² < 0.05, under 6 percent. The negative control is volume: MSE alone reaches amplitude 0.391 and IC 0.530, and CalibRank adds little: collapse never appears when predictability is sufficient.
For anyone doing quant work or multi-series decision problems, the paper hands over an actionable audit: estimate the target's predictability ceiling (fitted baselines' achieved R² is a lower bound); low ceiling plus a decision that depends on cross-series structure marks the danger zone. Changing architecture will not fix amplitude; adding a matched cross-sectional loss term is what moves ranking. CalibRank is a loss-function change that leaves the backbone untouched, so any existing training pipeline can adopt it. The broader warning is about evaluation: 43 of GIFT-Eval's 97 configurations are multivariate, but all are flattened into independent series before scoring, and none of its eleven metrics measures raw amplitude or cross-series structure. Per-series metrics can completely hide what downstream decisions actually consume.
The authors' own list: the amplitude identity is exact for best-scaled forecasts and checked in simulation, but the GIFT-Eval analysis only shows an association; achieved R² is a lower bound and the theorem's inequality is never tested directly. The cross-sectional diagnosis and remedy are demonstrated on one real panel; twelve backbones rule out a single-architecture explanation but not domain-specific ordering failures elsewhere. CalibRank targets one dependence functional (per-timestamp Pearson correlation); applications needing rank, tails, or calibrated scenarios need a different cross-sectional term, and it requires a meaningful cross-section at each timestamp, so single-series forecasting is out of scope. Two things stand out on a close read: the decision-return jump from 0.052 to 0.170 is gross of transaction costs, so live-trading implications should be discounted; and IC = 0.126 translates to a hit rate near 51.6%, meaningful in quant terms, but amplitude still runs nearly 2× off target, and deploying requires picking the operating point on a date-based validation split against an amplitude tolerance.