Quail adds hybrid-architecture support, caches recurrent state for >3x speedup over vLLM
sh_reya · x · 2026-11-17
Quail, an open-source inference engine from CMU's Full Stack Data Lab that runs AI functions (like AI.IF) inside SQL via Apache DataFusion, now supports hybrid-architecture models. With fewer attention layers, KV sizes shrink, improving cache reuse across queries. The team adapted its cost models and extended caching to save recurrent state, reporting a >3x speedup over vLLM in a new blog post. The MIT-licensed repo has 188 stars.
More from coding & agent
- Models earn 9x more from switching ideas than refining, but 58% never find the best basin — AlexGDimakis · 2026-11-17
- Opensteer: open-source local Mac agent powered by your Claude subscription — dinotimm · 2026-11-17
- Fable 5.2 spent 7 hours and 8 critic rounds building a drivable Fiat Uno game in one 603 KB HTML file — Viktorkav · 2026-11-17
- Poland's engineering teams have stopped writing code by hand entirely — PawelHuryn · 2026-11-17
- Fixing multi-user agent collaboration: one-click rooms debut in four-part release — thechrisperry · 2026-11-17
- Open-source Pascal Editor brings near-native 3D building design to the browser via WebGPU — anthara_ai · 2026-11-17