Quail adds hybrid GDN model support, beating vLLM by 3.2x on long-doc queries

sh_reya · x · 2026-11-17

Full Stack Data Lab extended the Quail inference engine to support hybrid-architecture models like Qwen3.5, where Gated DeltaNet linear-attention layers keep a fixed-size state (576 MB vs 2.36 GB KV for a 16k-token doc, 76% less). Quail now checkpoints GDN state alongside the KV cache so follow-up queries skip reprocessing, with revamped memory management and cost models — achieving a 3.2x geometric-mean speedup over vLLM on 46 QUAIL-B queries.

Related event: Quail open-source inference engine adds GDN hybrid architecture support, 3.2x faster than vLLM(2 posts)→

Original post →

More from Infra

Infra channel →