OpenDS-World benchmark: best of 12 frontier models solves only 39% of real-web deep research tasks
jiqizhixin · x · 2026-11-15
A team from HKUST (Guangzhou), Fudan, UW, Imperial College London and others introduced OpenDS-World, a benchmark of 200 real-web deep research tasks. The best of 12 evaluated frontier models answers just 39% correctly.
Key design details:
- Each task starts from a video clue; agents must discover, access and connect evidence across live websites, spanning interactive pages, audiovisual material, scientific figures and structured files
- Independently verified evidence chains support unique factual answers rather than predefined tool sequences
- All models share one tool environment with a budget of 3,600 seconds or 100 tool calls
Finding a page is only part of the challenge: picking useful sources and recovering from failed interactions remain major weaknesses for frontier models.
More from Models
- Why do LLMs always end replies with a question? Reddit users suspect engagement tuning — TaurusGold · 2026-11-15
- Chart shows Gemini 2.5 Pro sliding from LMArena #1 to #42 in about a year — Sauers_ · 2026-11-15
- Dev says Google Antigravity's Hydrogen model is "VERY good and VERY cost effective" — jasonkneen · 2026-11-15
- Grok 4.8 reportedly rolling out to Grok Bot users, unconfirmed yet — mark_k · 2026-11-15
- Llama 405B Base Model Natively Mimics o1's Chain-of-Thought from 128 Pages of Context — Sauers_ · 2026-11-15
- Gemini 4 'Hydrogen' Rumored Faster Than GPT and Claude, a Third Model Class — mertdumenci · 2026-11-15