Top-b sampling: entropy-adaptive decoding beats Top-p on GPQA and cuts variance 10x
cneuralnetwork · x · 2026-10-26
A new decoding strategy called Top-b uses Shannon entropy to dynamically adjust the candidate token band: narrowing it when the model is confident, expanding it when uncertain. Tested on Gemma 3 4B against Top-k, Top-p, Min-p, Typical and EAGLE-2, Top-b hit 26.80% on GSM8K (vs 26.13% for Top-p) while reducing run-to-run variance by 10x, and topped GPQA Diamond at 22.85% with the lowest average generation entropy (0.1579). Key insight: without retraining, just changing how you sample from the distribution makes generations far more consistent.
More from Models
- Bindu Reddy: US open source models are 'insanely behind' Chinese models — bindureddy · 2026-10-26
- Gemini 4 Flash spotted in Antigravity: internal build gemini-4-cp with 200k context — koltregaskes · 2026-10-26
- Depth-Grown and Looped LMs Learn In-Context Learning Earlier Than Dense Models — omarsar0 · 2026-10-26
- Current LLMs Are Horrible at Type Design in Dependent Type Languages — hargup13 · 2026-10-26
- Chart: Claude Opus 5.5 hits 34.4% of token spend within two weeks, chasing Opus 4.6's peak — scaling01 · 2026-10-26
- Developer: Claude Is Bad at Security Audits — letandrewcook · 2026-10-26