LoHoSearch turns a 7.62M-entity knowledge graph into a harder benchmark for search agents
美团技术团队 · wechat · 2026-07-23
Meituan's LoHoSearch uses a 7.62M-entity knowledge graph to generate harder search tasks
Meituan's LongCat team released LoHoSearch, a new benchmark for search agents that automates question generation from a large Wikipedia knowledge graph instead of relying on human-designed prompts.
- Built on a graph covering 7.62 million entities and 265 million directed edges.
- Contains 544 manually verified questions across 11 domains.
- Difficulty is controlled along two axes: search space size and structural complexity.
- The pipeline includes graph construction, subgraph sampling, question generation, automatic verification, and manual review.
Performance is much lower than on BrowseComp:
- The best model, GPT-5.5, scores only 34.74%.
- DeepSeek-V4-Pro, Claude-Opus-4.6, and Kimi-K2.6 cluster around 15.53%–15.99%.
- Other models are below 14%.
Additional findings:
- Solving a LoHoSearch task requires more tool calls on average: 61 vs. 35 on BrowseComp.
- Sampling helps, but even pass@16 tops out at 38.3%.
- Common context-management tricks improve only 6.8 points, far less than on BrowseComp.
The team argues LoHoSearch is a better stress test for long-horizon search and context management, and the benchmark is open sourced.
More from coding & agent
- Hermes Agent used as browser QA tester finds 8 issues in a bug-seeded site — techlatest_net · 2026-10-27
- Open-source FableCut lets AI agents edit a real video timeline via remote MCP — FastPresence9799 · 2026-10-27
- Redditor runs Hermes Agent in a dedicated Ubuntu VM to test real tasks — techlatest_net · 2026-10-27
- Developer drops VS Code, Office and git client, now uses only Claude — One_Improvement_6470 · 2026-10-27
- Grok bot makes its first real purchase, spots hidden prompt injection at checkout and ignores it — msg · 2026-10-27
- Swarmatory: Open-source shared chatroom/feed for your team's coding agents — qhoxie · 2026-10-27