Reasoning as pathfinding: RL boosts the odds of a correct path, not each chain
bibryam · x · 2026-11-20
Alex Wang's essay reframes LLM reasoning as pathfinding in token space: each sample is one attempt to find a path from the prompt to a correct answer.
Key points:
- RL training maximizes the probability of finding some correct path, not the correctness of individual chains of thought.
- Scaling RL (more samples, longer CoTs) raises success probability but increases reward hacking risk due to imperfect verifiers.
- The benefit of "thinking longer" at inference comes from compute invested during training that got amortized into the weights—individual chains don't converge to an optimum and can even degrade when forced longer.
The perspective also explains oddities like unfaithful chains of thought and overthinking.
More from Models
- Turing Post Author: Gemini's Neon Orchestrator Got Him Excited at 3 AM — TheTuringPost · 2026-11-20
- Ex-researcher: Gemini Can Now Run Post-Training Itself, Only Taste Is Missing — xennygrimmato_ · 2026-11-20
- GPT-6.2 Astra and Sol Stay on the Pareto Frontier Even With Capabilities Discounted — haider1 · 2026-11-20
- Google launches Gemini 4 Neon, a mid-range frontier-intelligence model built for code and agents — emmanuelvivier · 2026-11-20
- Screenshot leaks Gemini 4 Neon/Argon/Hydrogen lineup with Carbon unavailable, plus Claude Opus 5.5 — createyourwebsite · 2026-11-20
- Hands-on: Qwen4-Max shows big gains in frontend and agent skills over Qwen3.8-Max — karminski3 · 2026-11-20