QSO: A New HRL Method That Learns Direction-Clean Options Where Baselines Collapse
j_foerst · x · 2026-10-27
A shared thread introduces QSO, a new hierarchical RL method addressing a core HRL gap: in principle hierarchical policies should aid both credit assignment and transferability, but existing HRL algorithms either discard information needed for control or retain unnecessary information, limiting experience aggregation.
Accompanying figures show a maze-environment comparison of learned options across HIQL, OPAL, SHARSA/DSHARSA/ASO, and QSO. QSO's options cluster cleanly by action direction (up/down/left/right/dither) in PCA space, with a crisp block structure in the whitened option-distance matrix, while baselines show muddled, poorly separated option structures.
More from Research
- Penn's Kording Lab opens nominations for Spring 2027 speaker series — KordingLab · 2026-10-27
- neuron-explorer adds MLX backend for per-neuron steering on Apple Silicon — arthurcolle · 2026-10-27
- OpenAI says its models produced a Yang-Mills existence proof, formalized in Lean — exophades · 2026-10-27
- Bubeck hails subexponential-time 3SAT algorithm as biggest TCS surprise in 20 years — danintheory · 2026-10-27
- Greg Brockman announces AI-assisted discoveries in math, CS, and quantum field theory — gdb · 2026-10-27
- AI is simultaneously flooding journals with slop and cleaning up human slop — _akpiper · 2026-10-27