Join ordering, Kaplan-Meier decode estimates and trie attention yield 10x analytics inference speedup
charles_irl · x · 2026-11-12
- At a Modal × Full Stack Data Lab AI-SQL talk ("the unholy union of databases and inference") in San Francisco, the author presented three techniques to boost inference efficiency in analytics by 10x: join ordering, Kaplan-Meier estimates of decode length, and trie attention.
- The core idea imports database systems techniques into LLM inference: reordering execution like query planning, using survival analysis to predict decode lengths, and exploiting prefix structure via trie attention.
More from Infra
- Curated 9-paper reading list for LLM quantization beginners, from QLoRA to BitNet — East-Muffin-6472 · 2026-11-12
- llama.cpp PR #31915 fixes Gemma4 tool calling, user reports it "completely fixed" agentic coding — jacek2023 · 2026-11-12
- Nvidia board reportedly working with outside firm to find Jensen Huang successor — kimmonismus · 2026-11-12
- RDNA3-tuned llama.cpp fork doubles Qwen3.8-27B speed on RX 7900 XTX to ~100 tok/s — Jebbyk1 · 2026-11-12
- Ming Chi Kuo: Huang is irreplaceable to Nvidia; Rubin mass production slips to 2027 on HBM4 — zephyr_z9 · 2026-11-12
- Breakdown: Only 3% of Anthropic's 2.68M Chips Are Nvidia, Lowest in AI — teortaxesTex · 2026-11-12