Why BPTT was abandoned: control interface frequency drives gradient explosion
daibond_alpha · x · 2026-11-10
A technical thread on why backpropagation through time (BPTT) fell out of favor in RL. DreamerV2/V3 moved to REINFORCE-style gradients and robotics converged on PPO. The author argues the real change is the control interface: 50 Hz PD position control pulls errors back at every step, keeping BPTT gradients bounded, while direct torque control makes them explode (finger pointing at MuJoCo). Broader lesson: sometimes the algorithm isn't broken—the problem formulation blocks the solution.
Related event: SuperMimic Trains SOTA Humanoid Tracker in 3 Hours on a Single GPU(5 posts)→
More from Research
- 'Bitter lesson for robotics' gets a sharp rebuttal: it's still human-orchestrated end to end — burny_tech · 2026-11-10
- Video-Coder trains a coding agent for video generation, gaining 24+ points on PhyGenBench — HuaxiuYaoML · 2026-11-10
- IAS talk frames mathematics as a gift economy — fkasummer · 2026-11-10
- PSI claims proof of Aaronson-Ambainis conjecture using a research system built on Claude — burny_tech · 2026-11-10
- LingBot-Touch lifts VLA success rate from 45.6% to 72.5%, pretrained on 4,983 hours of tactile data — CyberRobooo · 2026-11-10
- How LingBot-Touch works: one shared Haptic Transformer unifies magnetic, piezoresistive and visuotactile signals — CyberRobooo · 2026-11-10