LoHoSearch turns a 7.62M-entity knowledge graph into a harder benchmark for search agents
美团技术团队 · wechat · 2026-07-23
Meituan's LoHoSearch uses a 7.62M-entity knowledge graph to generate harder search tasks
Meituan's LongCat team released LoHoSearch, a new benchmark for search agents that automates question generation from a large Wikipedia knowledge graph instead of relying on human-designed prompts.
- Built on a graph covering 7.62 million entities and 265 million directed edges.
- Contains 544 manually verified questions across 11 domains.
- Difficulty is controlled along two axes: search space size and structural complexity.
- The pipeline includes graph construction, subgraph sampling, question generation, automatic verification, and manual review.
Performance is much lower than on BrowseComp:
- The best model, GPT-5.5, scores only 34.74%.
- DeepSeek-V4-Pro, Claude-Opus-4.6, and Kimi-K2.6 cluster around 15.53%–15.99%.
- Other models are below 14%.
Additional findings:
- Solving a LoHoSearch task requires more tool calls on average: 61 vs. 35 on BrowseComp.
- Sampling helps, but even pass@16 tops out at 38.3%.
- Common context-management tricks improve only 6.8 points, far less than on BrowseComp.
The team argues LoHoSearch is a better stress test for long-horizon search and context management, and the benchmark is open sourced.
More from coding & agent
- Claude voice mode now calls external tools, reaches email and Notion, and deploys to Vercel — koltregaskes · 2026-07-23
- Netlify pairs Drop with Agent Runners for drag-and-drop site edits in minutes — thisiskp_ · 2026-07-23
- Lower inference costs make specialized production agents and evals practical — LangChain · 2026-07-23
- A CLI plus live-reloading UI workflow lets agents handle non-coding work — mattpocockuk · 2026-07-23
- AI engineering now has four overlapping layers: prompt, context, harness and loop — victorialslocum · 2026-07-23
- AI agents still need constant babysitting, or they quickly turn into spaghetti — IanArawjo · 2026-07-23