CypherGuard fixes structurally wrong LLM Cypher queries, hitting 88.5% on CypherBench

JeremyCMorgan · x · 2026-11-12

Problem: LLM-generated Cypher queries can be syntactically valid and schema-compliant yet return zero rows—e.g., traversing a relationship against its stored direction. On Cypher-Bench a small model scores only 31.4%, and even a strong model plateaus at 71.76%; dominant failures are structural, not linguistic.
Method: CypherGuard is a training-free framework with two core pieces: a graph-native error taxonomy guiding repair, and a mined rule library guiding query generation. At inference it first rewrites schema-detectable faults, then re-executes each draft and repairs only when the query errors or returns nothing.
Results: Across seven graphs on CypherBench test questions, CypherGuard reaches 88.50% overall execution accuracy—+25.08% over the same-backbone zero-shot floor—surpassing an RL-fine-tuned 7B specialist and the strongest training-based system compared.
Takeaway: A practical database-grounded validate-and-repair pattern for production Text2Cypher pipelines.

More from coding & agent