MEM-SQL: Memory-Empowered Text-to-SQL with HARE History-Aware Benchmark
hkuhk · hf · 2026-11-16
Researchers from HKU, NUS, and HKUST(GZ) introduce HARE (History-AwARe Text-to-SQL Benchmark) and MEM-SQL, addressing a blind spot in Text-to-SQL evaluation: widely used benchmarks score questions independently, ignoring real settings where earlier queries over the same database supply joins and calculations.
- HARE: 1,600 requests derived from a fixed pool of 823 verified queries across four datasets, with annotations recording which components each request preserves vs. edits—distinguishing whole-query adaptation from partial reuse.
- MEM-SQL: stores whole-query templates, reusable SQL blocks, and mappings from domain concepts to schema elements, consulting schemas on demand and reusing verified SQL components to improve accuracy while reducing cost.
The work makes a case for history-aware evaluation that better reflects incremental, real-world Text-to-SQL usage.
More from Research
- METR researcher: frontier labs dodge hard science evals, searching under the streetlamp — rbhar90 · 2026-11-16
- Stanford's build-an-LLM-from-scratch course CS336 publishes full lectures, slides and code — stanfordnlp · 2026-11-16
- SpeedRunBench test: Inkling-2-Small clears SuperTux 1-1 in 1,961 frames — simonguozirui · 2026-11-16
- Grid Robotics Proposes Agentic Robotics Paradigm Built on Embodied System 2 — Grid Robotics · 2026-11-16
- LLM Peer Reviewers Fixate on Appendices, Unlike Humans, Study Finds — CincyNLP · 2026-11-16
- StepRefine: Step-Aware Decoding Cuts Hallucinations in Multimodal Diffusion LMs — Cornell0917 · 2026-11-16