Microsoft Intern Project: Reusing Hidden States at Decoding Boosts LLM Performance for Free
burny_tech · x · 2026-08-13
A novel approach from a Microsoft AI Frontiers internship project enhances LLM performance by feeding the previous hidden state alongside the token embedding during decoding. This technique requires no architectural changes or additional parameters, effectively boosting model performance "for free".
More from Research
- Quiz: Double the Residual Stream from 4096 to 8192 — How Does Param Count Change? — Sauers_ · 2026-10-16
- Researcher credits GPT for multiple accepted papers, incl. disproving Wegner's 1965 conjecture — ctjlewis · 2026-10-16
- AutoAgent0: VLM-powered agentic runtime lifts long-tail driving success by 28.3pp — yining_hong · 2026-10-16
- Shuffled labels hit AUC 0.59: selection bias, not signal, drives prosody benchmarks — maier_ak · 2026-10-16
- CODESTRUCT reframes codebases as AST action spaces to fix brittle LLM code agents — burkov · 2026-10-16
- Score drift in RL fine-tuning is distillation toward a moving teacher, explains researcher — nrehiew_ · 2026-10-16