Freeform Preferences: Multi-Dimensional Robot Supervision
chelseabfinn · x · 2026-07-03
The author introduces "freeform preferences": supervisors first define relevant evaluation dimensions and then provide preferences along them. Dimensions can be fixed scoring criteria or natural language descriptions. This approach eliminates ambiguity, comprehensively covers all aspects, and provides denser supervision signals (method diagram included). The post serves as the core method explanation for their reward model research thread.
More from Research
- Exponential launches RepoInject and BrowserInject benchmarks for prompt injection robustness — maksym_andr · 2026-10-27
- Odd eval result: Gemini 4.1 Flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27
- Redwood: CoT controllability evals are under-elicited; better prompts boost scores 2-3x — dfrsrchtwts · 2026-10-27
- Inverse-designed optical system solves MNIST using BeamZ — S_Conradi · 2026-10-27
- DeCert supervises generative reward models' rationales without reference explanations — Guiquan Sun · 2026-10-27
- VLM harnesses act as compositional generalizers, new blog post shows — a1zhang · 2026-10-27