Salesforce open-sources GIFT-Eval-CTX: structured context cuts LLM forecast errors by 40.7%
LiJunnan0409 · x · 2026-11-11
Salesforce AI Research released GIFT-Eval-CTX, an open-source benchmark measuring how well LLMs and agentic AI turn forward-looking context (promotions, heatwaves, product launches) into precise numerical forecasts — information traditional time-series foundation models can't ingest.
- 10,000 tasks across 20 real-world datasets spanning energy, supply chain, IT and retail
- Structured context about upcoming events cut forecasting error by 40.7% for GPT-5.6 Sol and 29.6% for Claude Opus 4.8
- MoiraiAgent, pairing an LLM with Moirai-2 and code execution, achieved 36.3% lower error than the best history-only model, with its lead widening on longer horizons
Takeaway: feeding structured operational context into agentic AI drastically reduces error, turning static predictions into proactive planning. Paper and blog are public.
More from Models
- Dev reports Fable 5.2 works excellently for robotics tasks — R4nsack · 2026-11-11
- Leak claims Qwen4-27B lands Nov 11, 3pm Beijing time — unverified — teortaxesTex · 2026-11-11
- 4 Image Models, 5 Art Styles: Hands-on Comparison of MiniMax H3, Seedream 5.5, Nano Banana 2.1 and GPT Image 2.6 — aziz4ai · 2026-11-11
- Astra still rules high-complexity verifiable tasks two months on, Fable 5.2 wins interactive work — JoshPurtell · 2026-11-11
- Astra still king at verifiable coding tasks 2 months post-release, dev benchmarks show — JoshPurtell · 2026-11-11
- Blog: OLMo 3's verbalized eval awareness is mostly copied from SFT training data — BlancheMinerva · 2026-11-11