Quail's Draft-from-Document AI Extraction Is 4.7x Faster, Up to 19x
sh_reya · x · 2026-11-10
shreya and charlesirl published a new post on Quail, their open-source query engine with AI functions.
- Key idea: for AI-SQL inference workloads, answers often live in the queried document itself — so draft output from the document instead of generating from scratch.
- They implement adaptive spec decoding that learns per-query optimal draft lengths at runtime, with drafts up to 64 tokens.
- Result: their AI.EXTRACT implementation averages 4.7x faster (up to 19x) than a naive vLLM baseline, and they claim 2x faster and cheaper than other carefully-engineered inference solutions.
More from Infra
- 26 researchers publish 212-page paper on feasibility of a "hardwired pause" of frontier AI — DavidSKrueger · 2026-11-10
- Frontier lab compute shifts hard: pre-training falls from 67% to 7%, post-training/RL now 55% — akbirthko · 2026-11-10
- Study of 1,600 Data Centers Finds Each Adds ~1,000-1,300 Local Jobs Within 3-5 Years — RachelVT42 · 2026-11-10
- rakyll teases Agent Substrate as 'agentic compute fabric' ahead of KubeCon keynote — rakyll · 2026-11-10
- ngrok AI Gateway adds Liquid AI and 160+ LLM API models; tiny LFM3-1.7B pages for a thirsty plant — JosephJacks_ · 2026-11-10
- Together Compiler hits 2.7x throughput gains on Kimi K3 at same per-user speed — togethercompute · 2026-11-10