ORBIT Training Paradigm Boosts Zero-Shot Forecasting for Time Series Foundation Models
chaumian · x · 2026-08-16
This paper introduces ORBIT (Omni-Range Bootstrap Incremental Training), a training paradigm designed to control pre-training distributions for time series foundation models (TSFMs) on large-scale heterogeneous corpora.
Key Contributions:
- Bootstrap Multi-Level Sampling: Controls dataset exposure by sampling records, target variables, context windows, and prediction horizons.
- Omni-Range Incremental Training: Varies context lengths and prediction horizons within a single training stage.
- Falcon-2.0 Model: A univariate encoder-only Transformer trained under ORBIT, featuring missingness-aware triple-channel patch tokenization and parallel patch prediction.
- Rank-Guided Cross-Depth Alignment: A training objective using late-layer representations as stop-gradient teachers for shallow layers without extra inference cost.
Evaluations on GIFT-Eval and fev-bench demonstrate strong zero-shot forecasting performance across diverse domains and frequencies.
More from Research
- Study: Capabilities outside training scope hard to recover; K-5 model generalization limited — paulnovosad · 2026-08-16
- Exploring Visual Model Fingerprinting via Tensor Scanning — Glad_Contest_8014 · 2026-08-16
- Ofir Press: The industry is now driven by evals — OfirPress · 2026-08-16
- Study finds forced experience consolidation degrades LLM agent memory — _AndrewZhao · 2026-08-16
- Paper: Gender-Associated Linguistic Bias in LLMs — sbulaev · 2026-08-16
- CUHK's VideoCoCo: executable code as CoT lifts VBench-2.0 average by 25.7 points — 机器之心 · 2026-08-16