RL never uses uniform sampling: bias sources and the RFCL fix

KyleMorgenstein · x · 2026-11-19

Kyle Morgenstein explains that uniform sampling is never used in RL training—even vanilla RL—since trajectories are inherently biased by state and policy distributions. A very narrow reset buffer can indeed bias training, but approaches like RFCL can expand the distribution. The thread's earlier point: reset state choice biases the model, but good reset state generation is much easier than reward tuning.

Original post →

More from Research

Research channel →