First Text-to-SQL Model Beats Human Benchmark Using RLVR
EchoShao8899 · x · 2026-08-28
Researchers from UIUC and Bridgewater, in collaboration with Thinking Machines Lab, used Reinforcement Learning from Verifiable Rewards (RLVR) with expert-aligned data cleaning. This approach produced the first text-to-SQL model to surpass human performance (92.96%) on the BIRD benchmark. While frontier models like GPT-5.6 score in the mid-80s, they struggle with ambiguity and high costs.
More from Research
- Goodfire's Nick Cammarata: neural networks are "already clean" in a possible interpretability breakthrough — tszzl · 2026-10-15
- tszzl: interpretability breakthrough feels like my understanding of neural nets jumped five years — tszzl · 2026-10-15
- LightSuite: fast cell detection and whole-brain atlas registration on a standard workstation — hisspikeness · 2026-10-15
- Pathway CSO pitches BDH as post-Transformer architecture uniting memory, reasoning, adaptation — schwentker · 2026-10-15
- AI-designed molecular scissors cut Alzheimer's proteins — but is it really AI? — gerardsans · 2026-10-15
- EmbodiedEvo tops RoboDojo at 38.95% success by letting robots inherit failed practice — thetripathi58 · 2026-10-15