Reward Shaping Accelerates Robot RL Training by 5x

carlosdponx · x · 2026-08-08

The author demonstrates how reward shaping drastically impacts reinforcement learning (RL) efficiency through comparative experiments.

Initial Setup: Adding "look at target" observation and reward to a baseline task. After 3-4 days of training on an A100, the agent eventually learned the task and stayed at the target.
Optimized Setup: Refactored to use a polar observation scheme (where the agent simply minimizes certain values to zero) and a combined 1.0 weight reward incentivizing turning, moving, stopping, and maintaining arrival heading.
Results: Massive speedup. The agent learned the ideal behavior in less than half a day, compared to 2.5 days previously.

Key Takeaways: Shaping RLFT rewards yields huge gains for GPT-motion models. GPC is powerful because even with simple rewards, it eventually uncovers the "most natural" behavior from its existing priors (like "walk and stop"), essentially chiseling out existing skills rather than learning entirely new ones.

Original post →

Reward Shaping Accelerates Robot RL Training by 5x

More from Embodied