ANSER-Bench: 2,235 analytical search tasks trip up SOTA Deep Research in ~50% of cases
CShorten30 · x · 2026-10-31
A Tsinghua team (Qingyao Ai et al.) released ANSER-Bench, the first benchmark for analytical search — where models must produce a defensible conclusion rather than just retrieve in isolation.
- 2,235 bilingual (Chinese + English) tasks across law, finance, and scientific research, with evidence annotations and 14 task settings.
- Key finding: existing retrieval methods, including SOTA "Deep Research" baselines, fail in nearly 50% of cases despite high retrieval scores.
- Evaluated 8 baseline systems × 2 backbones.
- Site: anser.thuir.cn; paper, code, and data are open-sourced (GitHub: DavidMiao1127/ANSER-Bench).
More from coding & agent
- 'Don't use tools you can't fully control': OpenClaw pitch sparks agent autonomy debate — heyneighbor · 2026-10-31
- Dev builds Waifu Weather iOS app entirely with Claude Code, from Blender 3D modeling to music — MurkyGiraffe5879 · 2026-10-31
- Open-source /REA-growth skill reverse-engineers any company's growth playbook for $0.58 — _AustinCalvert_ · 2026-10-31
- Team runs Devin inside Slack and Linear as an autonomous project manager for mobile dev work — tomchapin · 2026-10-31
- OpenReceive ships open-source Laravel package for self-hosted Bitcoin and stablecoin checkout — Aiden_Tech_Ai · 2026-10-31
- Cloudflare Durable Objects No Longer Interrupted by Deploys, Update on Hibernate — threepointone · 2026-10-31