GPT-5.6 Luna reranking barely improves from top-20 to top-100 docs at 5x the cost
eshorten300 · x · 2026-11-19
Hands-on testing of listwise reranking with GPT-5.6 Luna shows the approach breaks down at scale: expanding the rerank window from top-20 to top-100 documents lifts nDCG@10 only from 0.470 to 0.482 — a +0.012 gain at 5x the cost.
The takeaway: long-context processing remains an open problem for LLMs, and search/reranking is one of the best probes for measuring progress on it.
More from Research
- New essay makes the case for traditional social science in studying AI agent interactions — RobbWiller · 2026-11-19
- If a model generates valid math blackboards, does it understand math? — francoisfleuret · 2026-11-19
- The Zvi on GTA 6's 99 Metacritic score: 'Welcome to 2026. We need better benchmarks.' — NathanpmYoung · 2026-11-19
- Stanford blood test ages 11 organs from 5,000 proteins; Helios launches 1,000+ biomarker home test — jgooten · 2026-11-19
- Kaggle lays out roadmap to fix AI benchmarks: relevance, saturation, verification, infrastructure — MeganRisdal · 2026-11-19
- Researcher: my real peer review is asking the world's top 2-3 experts, conference reviews are noise — taiyasaki · 2026-11-19