PreSTO: Predictive Subtree Prefetching Speeds LLM Power Sampling Up to 2.33x
morganstanley · hf · 2026-10-12
Morgan Stanley researchers propose PreSTO, a prefetching method that accelerates power sampling for LLM reasoning at test time.
- Problem: PowerMH's sequential Metropolis–Hastings updates make power sampling slow—each accept/reject decision sets the state for the next proposal.
- Method: PreSTO treats future MH outcomes as a tree of accept/reject decisions, batching proposals for a prefetched subtree per model call, provably leaving the MH transition kernel unchanged; a predictive traversal rule picks subtrees maximizing completed transitions.
- Results: across 7 datasets and 5 base LLMs, median speedups of 1.71x, 1.91x, and 2.33x on PowerMH, EntropyCut, and MultiTryMH; 3.9x fewer model calls (each only 2.1x longer) on the first two; the traversal rule adds 1.25–1.37x on LCB V6 with Qwen3.5-9B.
Related event: PreSTO Speeds Up LLM Power Sampling by Up to 2.33x(2 posts)→
More from Infra
- Lumentum Optics Sold Out Through Early 2029, Yet Only 44 of 149 AI Stocks Rose Last Week — demian_ai · 2026-10-12
- HF Engineer Runs pi-web on DGX Spark to Steer Local AI Agents From His Phone — lhoestq · 2026-10-12
- Open-source leaderboard whatmodel.sh shows the best local models your machine can run — jakecoolguy · 2026-10-12
- Running Qwen3.8-Flash-Next on six 3090s: 7 users at 262k context with a patched vLLM — flynth92 · 2026-10-12
- Grid becomes AI's new bottleneck: Entergy warned Amazon of peak-time cutoffs, AWS built Project Falcon — kimmonismus · 2026-10-12
- 'AI always maxes out your GPU'? Devs push back on the myth — mgostIH · 2026-10-12