Removing negative-advantage samples drops robot zip-tie success from 86% to 60%

micoolcho · x · 2026-11-08

@fancyyzc's team proposes TRFO, showing that learning from negative experience measurably matters for robot policy training.

Excluding negative-advantage samples cuts zip-tie success from 86% to 60%;
Removing the trust-region constraint causes training to diverge;
With the same collected experience, TRFO outperforms HG-DAgger and RECAP on laundry folding and zip-tie threading.

Conclusion: both the negative learning signal and constraining its strength contribute to effective policy improvement.

More from Embodied