Berkeley open-sources Quail, an ultra-high throughput AI-SQL engine with query plans
sh_reya · x · 2026-09-25
UC Berkeley's Full Stack Data Lab released Quail, an MIT-licensed ultra-high throughput AI-SQL engine built with Charles Frye of Modal, with a blog post, code, and playground demo.
- Context: AI-SQL (LLM-powered SQL functions, now offered by Snowflake Cortex, BigQuery, Databricks, MotherDuck) can trigger hundreds of thousands to millions of LLM calls per query, making it extremely expensive.
- Prior work: Academic systems like DocETL (Berkeley), LOTUS (Stanford), Palimpzest (MIT), ThalamusDB (Cornell) cut calls via optimizations like MOAR, Task Cascades, Abacus and cheaper-model routing (BARGAIN).
- Quail's key idea: organize execution around query plans; the new release adds AI filters and joins.
- Known gap: on agent-trace queries with matching prefixes, vLLM's automatic prefix cache is 2.32x faster — prefix caching is on Quail's roadmap.
More from Infra
- Modern Microprocessors: A 90-Minute Guide Still the Best Crash Course for Systems Engineers — blaizedsouza · 2026-09-25
- GPU returns hit +76% a year as H100 rents jump 49% and B300 rates climb 66% — KyeGomezB · 2026-09-25
- openjev-sglang: SGLang Radix Cache lets Qwen3.6-35B-A3B run 64 Jev decisions in under 1s — multiply_matrix · 2026-09-25
- Ramp benchmarks Jev to replace LLM reranking: 10x lower tail latency at 300ms, 3x cheaper — multiply_matrix · 2026-09-25
- Qualcomm pitches the phone as the AI hub at Snapdragon Summit, aiming for Apple-like cross-device experience — BenBajarin · 2026-09-25
- Running Qwen3.8-Flash-Next on a 5090 with llama.cpp: 40 tok/s and barely any RAM used — nirurin · 2026-09-25