Meta's hLLM decodes full rankings in O(1) passes with the Hungarian algorithm, 64x faster at 28ms
_reachsumit · x · 2026-09-03
Meta's hLLM (Hungarian LLM) tackles the decoding bottleneck in generative reranking: instead of autoregressively emitting N ordinals, it reads an N×K item-position score matrix off prefill hidden states with a lightweight attention head and solves optimal bipartite assignment via the Hungarian algorithm, yielding a valid permutation by construction. With LoRA fine-tuning and teacher-ranking distillation, it reaches 28ms end-to-end inference—a 64x speedup—at teacher-level ranking quality, with full ablations.
More from Research
- Stanford researcher clashes over genomics foundation model's cherry-picked variant prediction benchmarks — anshulkundaje · 2026-10-21
- Gemini Agents Crack 59-Year-Old Fusion Math Conjecture in Hours — DynamicWebPaige · 2026-10-21
- Bombadil: a foundation model that decodes plants' bio-electrical signals — tdhopper · 2026-10-21
- TrackEverything: CMU and Meta open-source a 3D tracker for all points in 1000+ frame videos — AdamWHarley · 2026-10-21
- Infantconnect: open-source infant brain anatomy and EEG connectivity tool from 1 to 72 months — Dr_Alex_Crimi · 2026-10-21
- Berkeley wins Schmidt Sciences HAVI grant to decode whale and parrot communication with AI — begusgasper · 2026-10-21