Open-source AI-SQL engine Quail hits 1B+ input tokens/min on a single H100
sh_reya · x · 2026-09-25
shreya's team, in collaboration with Modal, released Quail, an open-source AI-SQL engine that lets users write queries like SELECT FROM reviews WHERE AI.IF("the review discusses the ending").
The catch: AI operators are wildly expensive — an AI filter calls an LLM for every row, and an AI join for every candidate pair (mn), so one query can trigger millions of LLM calls.
By planning queries and LLM inference together, Quail reaches 1B+ input tokens/min on a single H100 for a single query — a new and interesting inference workload, the authors argue.
More from coding & agent
- Agents can scrape Docsend decks at will, so just turn on downloads by default — soleio · 2026-09-25
- Docker launches Cloud Sandboxes: microVM isolation for always-on agents, $250 credit — juntao · 2026-09-25
- Vercel AI SDK to add OpenAI functions support with three UX flows — maxleiter · 2026-09-25
- Developers now prompt AI even to change a CSS color code — saheedniyi_02 · 2026-09-25
- GPT-6 "Astra" agent clears three chapters of Left 4 Dead 2 with 0/5 escapes — imjustnewatai · 2026-09-25
- Perplexity launches Rust-based Fast Search: 95% of results in 230ms, Shopify adopts it — AravSrinivas · 2026-09-25