TREC 2026 RAG Track Opens: Featuring New Nvidia ClimbMix Corpus
lintool · x · 2026-08-06
The TREC 2026 Retrieval-Augmented Generation (RAG) track is now open, shifting to an "Agent-first" evaluation mode this year. The benchmark introduces a new corpus, ClimbMix-400b sponsored by Nvidia, replacing the traditional MS MARCO v2.1.
Participants can access the new corpus via the Pyserini REST API and compete in either Retrieval or RAG tasks. The track also released the RAGDoll evaluation toolkit, supporting an end-to-end workflow from generating evaluation artifacts to scoring LLM responses. Submissions are due by August 8.
More from Research
- Goodfire AI's MAPS Explains 2.1 Million Genetic Variants Mechanistically — mathildepapillo · 2026-08-06
- New Theory Explains the Effectiveness of Stop-Gradient in Flow Models — kwangmoo_yi · 2026-08-06
- Fudan Researchers Show AI Models Can Autonomously Self-Replicate Like Worms — willknight · 2026-08-06
- Evaluating LLM Sycophancy: Which Models Hold Their Ground? — zero0_one1 · 2026-08-06
- Why Models Generalize Coarsely When Put in a 'Bad' Context — nptacek · 2026-08-06
- GEPA Optimization Loop Gets 3-4x Faster With Parallel Candidate Evaluation — dbreunig · 2026-08-06