Biomedical benchmark: prewritten Cypher MCP tools hit 98% vs 85% for Text2Cypher
JeremyCMorgan · x · 2026-11-05
The open-source BiomedQA benchmark tests agent data access over three merged biomedical knowledge graphs with 40 pharmacology questions. Prewritten Cypher MCP tools scored 39/40 (98%, 920ms avg latency), beating Text2Cypher (34/40) and standalone GPT-4o (30/40). Not perfectly controlled, but a strong case for constrained tools when question patterns are predictable.
More from coding & agent
- "Hand-writing code will sound like a fever dream to our children," dev predicts — tlakomy · 2026-11-05
- Hybrid search plus reranker cuts retrieval failures 67%: Obsidian AI search guide — dSebastien · 2026-11-05
- Agent harnesses aren't dying: harness distillation changes the job, says O'Reilly — bibryam · 2026-11-05
- Anthropic shares Claude Code trick: run Sonnet, Haiku and Opus as builder, scout and advisor — Arindam_1729 · 2026-11-05
- Using JBang to Add Deterministic Java Guardrails to Your Coding Agent via Lifecycle Hooks — myfear3 · 2026-11-05
- Open-Source Research Agent Shows Calibrated Confidence Scores and Real Citations Instead of Fake Certainty — Bubbly-Tradition-735 · 2026-11-05