Q2D-Web: new retrieval benchmark with 190M web docs and 70k LLM-rewritten queries
antoine_chaffin · x · 2026-09-10
Researchers (antoinechaffin of LightOn) released Q2D-Web, a large-scale retrieval benchmark targeting web search — a use case existing benchmarks evaluate poorly.
- 190M web documents, with a subsampled split preserving ranking fidelity.
- 70k ecologically valid queries rewritten by LLMs (now among the largest sources of search queries), with deep RAG- and retrieval-oriented relevance annotations.
- Comes with a hosted leaderboard.
Related event: Perplexity Releases Q2D-Web, an Open Benchmark for Agentic RAG Retrieval(10 posts)→
More from Research
- JockeyBench uses Japanese horse racing to test LLM agents — all betting strategies lose money — Zhongwei Xie · 2026-10-27
- Stanford's PaperFormalizer turns research papers into Lean-verified versions with author in the loop — StanfordUniversity · 2026-10-27
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- Hugging Face TRL makes score centering + masked importance sampling default in async GRPO — _lewtun · 2026-10-27
- EMNLP 2026: sub-billion-parameter LMs show superhuman entity tracking, human-like fleeting memory — GretaTuckute · 2026-10-27
- 10-year study documents first 'unnatural gradient' phonotactic restriction in Tarma Quechua — begusgasper · 2026-10-27