Perplexity Releases Q2D-Web: 190M-Doc Benchmark for Agentic RAG Retrieval
_reachsumit · x · 2026-09-09
Perplexity AI introduced Q2D-Web, a large-scale benchmark for first-stage retrieval in agentic RAG, pairing a 190M-document web corpus with 70k agent-reformulated queries drawn from production user queries across ten languages. It addresses gaps in existing benchmarks, which either have huge corpora but few queries, or many queries but small corpora, and typically test human-written queries rather than machine reformulations. Three relevance judgment sets are provided (agent citations, production rankings, and a union augmented with LLM judgments). Benchmarks across 13 lexical, dense, and late-interaction retrievers show rankings are largely insensitive to judgment-set choice.
Related event: Perplexity Releases Q2D-Web, an Open Benchmark for Agentic RAG Retrieval(10 posts)→
More from Research
- AutoResearch agent produces weird 'forking' loss curves, and the team dug into the science — ZimingLiu11 · 2026-10-27
- No π0 weights changed: Motoniq layer lifts real-robot precision fitting from 0/10 to 9/10 — hbouammar · 2026-10-27
- JockeyBench uses Japanese horse racing to test LLM agents — all betting strategies lose money — Zhongwei Xie · 2026-10-27
- Stanford's PaperFormalizer turns research papers into Lean-verified versions with author in the loop — StanfordUniversity · 2026-10-27
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- Hugging Face TRL makes score centering + masked importance sampling default in async GRPO — _lewtun · 2026-10-27