ReDSI: First Open-Source DSI Implementation Covering All Three Document Identifier Types
_reachsumit · x · 2026-09-09
Researchers release ReDSI, the first open-source implementation of the differentiable search index (DSI) framework covering all three document identifier types (atomic, naive, semantic), along with a parameterizable, well-documented NQ320K construction pipeline.
Key points:
- DSI (Tay et al., 2022) is the de facto baseline for generative retrieval, but has been hard to reproduce: no public implementation covered all identifier types, reported results vary widely, and NQ320K preprocessing is underspecified.
- ReDSI achieves results competitive with or stronger than prior DSI baselines.
- Extensive downscaling experiments cover retrieval effectiveness, parameter efficiency, training methods, and decoding strategies, opening directions for future research.
Code and paper are openly available.
More from Research
- Timothy Duff's ECCV 2026 SfM-DL workshop slides on algebraic optimality for minimal solvers — ducha_aiki · 2026-09-09
- Drop a fixed batch proportion instead of per-sample tokens: capi author shares training trick — giffmana · 2026-09-09
- Adding Greek to a Cosmos3 VLA policy: bilingual training helps but lags far behind English — KIEFERSA · 2026-09-09
- Transformers encode a partner's expertise early but only act on it in later layers — Mika Okamoto · 2026-09-09
- Cadence uses a time-series foundation model for error-bounded lossy compression of demand data — Roberto Tacconelli · 2026-09-09
- Fourth Perception Test Challenge at ECCV 2026 pushes multimodal models on city-scale spatial intelligence — AjdDavison · 2026-09-09