CURE-UCB: Using Known Time Horizons to Pick Better Options in Rising-Reward Settings
burkov · x · 2026-11-17
Andriy Burkov highlights a research paper on sequential decision-making in rising-reward settings, where an option's payoff improves with repeated investment. Whether to commit to a slow-improving option depends on the remaining time horizon: great for a long training budget, poor near a deadline.
The paper's main contributions:
- Measures the value of horizon awareness — quantifying how much better decisions get when the known time horizon is used.
- Proposes CURE-UCB, an algorithm that estimates each option's likely cumulative reward over the remaining time, rather than judging only by current reward or a single future-point projection.
- Provides mathematical analysis and empirical comparisons against rising-reward and general non-stationary decision-making baselines.
The setting arises in tasks like model selection and machine-learning training, where investing in a slower-improving option can pay off only if there's enough time left.
More from Research
- MIT's Jacob Andreas trains models to explain their own computations — MacrinePhD · 2026-11-17
- ICLR reviewer flags AI-written reviews: a third of his batch undisclosed — vaiter · 2026-11-17
- IFM releases K2-Horizon: six fully open LLMs up to 375B MoE, trained on ~10T synthetic tokens — IFM · 2026-11-17
- PKU & Tencent Hunyuan's MovieRoll extends streaming video generation to 60s with 78% KV-cache savings — Yunyang Ge · 2026-11-17
- UMM-JEPA: predicting continuous perceptual representations beats discrete codes for unified multimodal models — Yucheng Zhou · 2026-11-17
- ScaleVLA scales VLA models to 552B parameters: 10 demos beat 2B backbone trained on 50 — GalaxyGeneralRobotics · 2026-11-17