Perplexity Releases Q2D-Web: A Benchmark with 190M Documents and 70k Agent-Reformulated Queries
antoine_chaffin · x · 2026-09-10
Perplexity introduces Q2D-Web (Query2Doc-Web), a benchmark and public leaderboard for evaluating retrieval in agentic RAG systems. Unlike common setups that focus on nDCG@10, it targets Recall@1000, measuring how embedding models perform on large-scale web search.
The benchmark scales up the earlier Q2D benchmark (released alongside pplx-embed-v1) by 7x, featuring 70k agent-reformulated queries and a 190M-document web corpus. The team hopes it becomes the modern "MS MARCO".
Related event: Perplexity Releases Q2D-Web, an Open Benchmark for Agentic RAG Retrieval(10 posts)→
More from Research
- JockeyBench uses Japanese horse racing to test LLM agents — all betting strategies lose money — Zhongwei Xie · 2026-10-27
- Stanford's PaperFormalizer turns research papers into Lean-verified versions with author in the loop — StanfordUniversity · 2026-10-27
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- Hugging Face TRL makes score centering + masked importance sampling default in async GRPO — _lewtun · 2026-10-27
- EMNLP 2026: sub-billion-parameter LMs show superhuman entity tracking, human-like fleeting memory — GretaTuckute · 2026-10-27
- 10-year study documents first 'unnatural gradient' phonotactic restriction in Tarma Quechua — begusgasper · 2026-10-27