Stackelberg Alignment turns LLM disagreement into a training curriculum via bandit leader
shangbinfeng · x · 2026-10-20
Researchers propose Stackelberg Alignment, built on the insight that when LLMs disagree, that's exactly where they learn most. The framework frames multi-LLM collaborative post-training as a Stackelberg game: an EXP3 bandit leader selects prompts rewarded for difficulty and discriminativeness, while follower models respond, judge each other with Elo-style reputation weighting, and learn via DPO/GRPO. Without difficulty labels, the leader induces an emergent curriculum — top-5 prompts rise to 1.6× uniform weight on reasoning data. Paper and code are public.
More from Research
- mjfit: open-source convex decomposition for tight-clearance parts, 10x faster than SDF — yacineMTB · 2026-10-20
- iSEE world model learns object permanence from video alone, no labels — CSProfKGD · 2026-10-20
- TasteBench tests model research taste on ever-fresh out-of-sample ML tasks — marktenenholtz · 2026-10-20
- "Neurons aren't already clean — interpretability just covers the dunghill with snow" — 1a3orn · 2026-10-20
- Single neuron steering makes Qwen3-8B insist it's an owl — Sauers_ · 2026-10-20
- Q-CTRL: over 99.5% of qubits sit idle in one fault-tolerant Shor's algorithm blueprint — MJBiercuk · 2026-10-20