CMU lab open-sources Quail, a SQL inference engine 1.8x faster than vLLM and up to 14x on AI joins
sh_reya · x · 2026-10-25
CMU's Full Stack Data Lab released Quail (QUery-Aware Inference Layer), an MIT-licensed open-source engine that runs LLM-powered AI functions inside SQL over large datasets on your own GPUs (171 stars on GitHub).
Key points:
- Optimizes LLM inference and memory management per query as a whole, rather than row-by-row
- Benchmarked at 1.8x faster on average than well-tuned vLLM baselines, up to 14x on queries with AI joins
- Supports SQL-native functions like AI.EXTRACT for extraction, classification, and AI joins over unstructured data
- Built on Arrow and vLLM, aimed at embedding LLM inference directly into analytics workflows
Author shreya demos using it to upgrade Slack-based data workflows.
More from coding & agent
- Making AI-animated series: which tools manage multi-shot video workflows beyond ComfyUI? — dtdisapointingresult · 2026-10-25
- Nutlope shares his 5-step vibe coding workflow: voice prompts, grill-me skill, verification recordings — nutlope · 2026-10-25
- YC-backed AI browser Aside claims #1 on three browser agent benchmarks — brandon_galang · 2026-10-25
- CritPt, SciCode and Terminal-Bench Science: three science benchmarks as RL environments — geoffwolfe · 2026-10-25
- REA reverse-engineering agent framework hits #1 on GitHub Trending with full learning roadmap — blaizedsouza · 2026-10-25
- BeautifulRust: a Claude Code plugin distilled from Google's Rust style guide for idiomatic agent code — blaizedsouza · 2026-10-25