schemagate + Claude Opus 5.5 Takes #1 on BIRD Mini-Dev Text-to-SQL Leaderboard
Temporary_Papaya1058 · reddit · 2026-10-26
The open-source schemagate entry was merged into the official BIRD Mini-Dev leaderboard: paired with Claude Opus 5.5 it scores 81.00 SQLite / 72.40 MySQL / 72.60 PostgreSQL execution accuracy—#1 on all three tracks (Gemini 2.5 Flash combo ranks #2), using one Claude call per question. The key layer selects only the tables/columns the question needs (respecting read permissions), then validates generated SQL against that schema before execution. Code, predictions, and scorer logs are open-sourced; the author offers to discuss setup and prompts.
More from coding & agent
- UserArena benchmark uses 64,729 real shopping trajectories to test long-horizon agents — Jujia Zhao · 2026-10-26
- ByteDance and universities open-source RSIHub, a composable agent-evolution framework — ByteDance · 2026-10-26
- Claude Opus builds an interactive map of all US water use, county by county — AndyMasley · 2026-10-26
- Tencent proposes TGR-EvoDAG, a DAG-based multi-agent code evolution framework — _reachsumit · 2026-10-26
- Writing evals may become AI's highest-paying job: whoever owns the rubric controls agent output — hwchase17 · 2026-10-26
- Perspective paper: Agentic memory management is the next frontier of information retrieval — _reachsumit · 2026-10-26