browsecomp-plus-mini: a 15-query subset that ranks deep-research agents like the full 830-query benchmark
asankhs · reddit · 2026-10-18
The author built a 15-query subset of BrowseComp-Plus to compare search/deep-research agents without running the full 830-query benchmark every time.
Key results:
- Weighted Spearman rank agreement of 0.88 with the full benchmark, vs 0.75 for a random 15-query subset
- Runs more than 10 points apart on the full benchmark keep their order 96% of the time; 5-10 points apart, 75%
- Answers are all 1-4 words, graded by normalized exact match — no LLM judge needed
- Costs under 2% of a full run
Dataset released on Hugging Face (LocalLLaMA/browsecomp-plus-mini).
More from coding & agent
- Laid-off geophysicist built a Claude Code job-search system, landed AI engineer role; repo passes 50k stars — techNmak · 2026-10-18
- Scrapling v0.5 lets AI agents auto-solve hCaptcha, reCAPTCHA, Turnstile and more — VeryWellVersed · 2026-10-18
- Paul Graham amplifies dev's plea: under 48-hour attack waves, defense needs 24/7 agents too — mhmazur · 2026-10-18
- Hermes agent plugin makes real WhatsApp voice calls, powered by Gemini Live — alexcovo_eth · 2026-10-18
- Agent + OpenVoice dubs entire movies into Polish on a 16GB M4 Mac mini — Teknium · 2026-10-18
- Dead ends are the costliest thing your AI agents keep relearning — Asly97 · 2026-10-18