Robotics RL is brutally hard: one practitioner's list of a dozen failure modes
Scobleizer · x · 2026-09-19
A robotics RL practitioner's candid retrospective: pattern matching isn't enough. Their run improved 2% then crashed to 100% failure — the KL term too low for the learning rate, the critic's gradient too high. The physics engine was too inaccurate for grasping, tactile sensors produced NaNs, sensors made training painfully slow, and sim frequency mismatched real-world inputs raising sim2real doubts. At one point entropy was zeroed by a simple reward bug, making "do nothing" optimal for three days before anyone noticed. Add an fp32 vs fp64 replay discrepancy, plus days lost to wrong answers from their AI assistant.
More from Embodied
- INMO Future glasses: dual 1080p displays, 125g, $799 early-bird on Kickstarter Dec 10 — Scobleizer · 2026-11-04
- DISCO distilled tree search named CoRL 2026 best-paper finalist, top 6 of 687 — yining_hong · 2026-11-04
- Berkeley's Value Forcing: enforcing Bellman consistency at every noise level for accurate RL value functions — berkeley_ai · 2026-11-04
- 22 Creative SAM 3.1 Use Cases From Meta's AI Developer Hackathon, From Wildfire Detection to Document Reassembly — Vjeux · 2026-11-04
- Ex-OpenAI robotics intern breaks down viral Astra robot-painting demo on Cerebras podcast — AccBalanced · 2026-11-04
- Embodied Agent Arena: Closed-Weight VLMs Lead All Five Robot Task Domains, Open-Weight Models Lag — Haojian Huang · 2026-11-04