RLVR with Cleaned Data Yields First Text-to-SQL Model to Beat Humans on BIRD

Bridgewater AIA Labs and UIUC researchers, collaborating with Thinking Machines Lab, published new work showing that RLVR (reinforcement learning on verified data) combined with a careful data-cleaning pipeline produced the first Text-to-SQL model to beat human performance on the BIRD benchmark at 92.96%, with inference costs up to 8x lower. The work also revisits the debate over whether RLVR is robust to noisy data.

Confirmed

The team released BIRD-Platinum, a high-quality Text-to-SQL dataset with label noise fixed (@ddkang)
Kimi-K2.6 trained only on BIRD-Platinum substantially outperforms tested frontier models including Sol and Fable; training used the Tinker API (@ddkang)
The paper fine-tunes Qwen3-235B via RLVR on clean data, reaching human-level Text-to-SQL performance without heavy engineering (@ddkang)
With a stricter cleaning pipeline, the team shows noisy data drops test accuracy by over 9%, and algorithmic improvements hardly mitigate this (@ddkang)

Unconfirmed

The study rebuts the earlier claim that '100% wrong data still works,' attributing it to contamination by correct labels—an explanation awaiting further community validation

Why it matters

This is the first Text-to-SQL model to surpass humans on BIRD (92.96%) at far lower cost, suggesting data quality, not model scale, may be the current bottleneck
If RLVR is noise-sensitive, prior RL training results on noisy datasets may need re-evaluation
The open BIRD-Platinum dataset offers a more reliable benchmark for future Text-to-SQL research

2026-08-28 ~ 2026-08-28 · 6 related posts

Primary sources

Study claims Text-to-SQL model achieves human-level accuracy at lower cost — ddkang · 2026-08-28 02:03 (UTC+08:00)
Kimi-K2.6 trained on BIRD-Platinum outperforms frontier models — ddkang · 2026-08-28 02:03 (UTC+08:00)
Human-Level Text-to-SQL via RLVR and Verified Data Cleaning — ddkang · 2026-08-28 02:07 (UTC+08:00)

Study: Noisy data severely degrades RLVR training performance — ddkang · 2026-08-28
[source] Study claims Text-to-SQL model achieves human-level accuracy at lower cost — ddkang · 2026-08-28
[source] Kimi-K2.6 trained on BIRD-Platinum outperforms frontier models — ddkang · 2026-08-28
[source] Human-Level Text-to-SQL via RLVR and Verified Data Cleaning — ddkang · 2026-08-28
Cleaning RL dataset boosts Text-to-SQL performance significantly — joecole · 2026-08-28
First Text-to-SQL Model Beats Human Benchmark Using RLVR — EchoShao8899 · 2026-08-28