A VLDB 2026 paper finds over 50% annotation errors in BIRD and Spider 2.0-Snow
ddkang · x · 2026-07-28
- A VLDB 2026 paper on text-to-SQL agents reports more than 50% annotation error rates in BIRD and Spider 2.0-Snow.
- The authors say those errors can change measured performance by up to 19%.
- The result is framed as a warning that benchmark and leaderboard numbers may be far less reliable than they appear.
More from Research
- A deep dive on building frontier-lab evals explains why 100% scores can be a failure — aakashgupta · 2026-07-28
- Maker shares first AI robot kit built with Raspberry Pi 5 and Hermes agent — petrusenko_max · 2026-07-28
- Exploring Artificial Life: Wolfram and Others Feature in Lenia Simulation — max_romana · 2026-07-28
- A new artificial-life video asks what’s missing for open-ended evolution — max_romana · 2026-07-28
- Macrocosmos starts a permissionless 16B model training run across three continents — markjeffrey · 2026-07-28
- Free AI curriculum maps a practical path from first principles to LLMs — tetsuoai · 2026-07-28