Berkeley's Value Forcing: enforcing Bellman consistency at every noise level for accurate RL value functions
berkeley_ai · x · 2026-11-04
Sergey Levine's group at UC Berkeley introduced Value Forcing, applying the flow matching recipe to RL value functions.
- Prior attempts to use flow matching for value functions produced biased, collapsed predictions
- The key finding: enforce Bellman consistency not just on the final values, but at every noise level of the value-generation flow process
- This yields an accurate value estimator even with very small amounts of data, producing value distributions that better capture the true distribution of values and leading to better policies
Chongyi Zheng notes flow matching has been very successful for robotics and RL policies, and this work extends the same recipe to value estimation.
Related event: Berkeley Team Introduces Value Forcing for Value Function Learning(2 posts)→
More from Embodied
- 22 Creative SAM 3.1 Use Cases From Meta's AI Developer Hackathon, From Wildfire Detection to Document Reassembly — Vjeux · 2026-11-04
- Ex-OpenAI robotics intern breaks down viral Astra robot-painting demo on Cerebras podcast — AccBalanced · 2026-11-04
- Embodied Agent Arena: Closed-Weight VLMs Lead All Five Robot Task Domains, Open-Weight Models Lag — Haojian Huang · 2026-11-04
- PrimeBot's PrimeSim: a generalist world model simulator trained on cross-embodiment demos — Travis Davies · 2026-11-04
- Physical AI Hackathon in SF Nov 7–8 features Dream-1 world action model and robot arms — Scobleizer · 2026-11-04
- Figure robot office demo looks like a sci-fi movie, says CEO Brett Adcock — adcock_brett · 2026-11-04