MARTINI algorithm mixes surrogate rewards to steer generative models without querying the oracle
mmbronstein · x · 2026-10-21
A new arXiv paper, "Steered Not Stirred: Effective Reward Tilting using Surrogate Reward Mixing," introduces MARTINI: given a small feedback set of oracle rankings or rewards, it learns a composition of multiple imperfect surrogate rewards to steer pretrained continuous-time generative models (diffusion, flow matching, flow maps) toward a ground-truth reward that's too expensive to evaluate at inference. The theory characterizes surrogate information limits and links population recovery error to ideal-target KL at small tilt strengths.
Related event: MARTINI Steers Generative Models at Inference Time via Mixed Proxy Rewards(3 posts)→
More from Research
- TUM paper: LLMs implicitly ration thinking tokens across composed tasks, causing errors — TUM · 2026-10-21
- Meta's OverSeer uses fixed auxiliary camera views to stabilize multi-agent world models — meta · 2026-10-21
- OpenAI invests in startup using brain implants to detect cancer, Wired reports — wiredmagazine · 2026-10-21
- Hugging Face Launches SimReady Central: Open Sim-Ready 3D Assets for Robotics — Hugging Face Blog · 2026-10-21
- GPT Image 2.5 vs Nano Banana 2.1: 502 synthetic rust masks reviewed, 79.7% retained — Gloomy_Recognition_4 · 2026-10-21
- NVIDIA cuDSS hits GA: GPU sparse solvers deliver up to 10x speedups in engineering simulation tools — NVIDIA Blog · 2026-10-21