ZeroGR paper: filtering-first decoding speeds generative recommendation up to 9.9x

_reachsumit · x · 2026-11-05

ZeroGR is a filtering-first constrained decoding system for generative recommendation. Since most candidate item sequences are invalid, scoring before filtering wastes compute; ZeroGR filters invalid sequences before scoring with negligible quality loss.
Key techniques: a step-sliced filtering index that tailors data representations per decoding step to cut GPU memory; a shared-slot execution engine with hybrid projection to accelerate stage data exchange; and layout-guided streaming construction for fast index refreshes with limited serving interference.
Evaluations show up to 9.9x speedup over representative baselines for the filtering/scoring pipeline, plus lower index memory use and faster index reconstruction.

More from Infra