Flide: exact state forking speeds decision-model agents up to 9.5x
NationalUniversityofSingapore · hf · 2026-11-10
NUS and Oxford researchers released Flide, an exact inference engine for decision-model LLM agents. It decouples the logical request tree from physical execution and uses ForkAttention with GPU-resident indices to guarantee every branch evaluates identically to an isolated forward pass, across both standard attention and recurrent Gated DeltaNet layers. Evaluations show up to 9.5x faster queries than vLLM and SGLang while staying closest to a float32 reference in total variation distance. Code is open source.
More from Infra
- simdjson disables 256-bit LASX on Loongson 3A6000: 128-bit LSX is 6.7% faster — lemire · 2026-11-10
- Vera launches gyms, benchmarks and agents for cloud infrastructure operations — tianyin_xu · 2026-11-10
- NVIDIA paper: masking ~10 'intrusion heads' recovers 57% of distractor accuracy loss — rohanpaul_ai · 2026-11-10
- Memory pricing itself out of demand: PC BOM share hits 35-38%, shipments plunge — tengyanAI · 2026-11-10
- Two engineers burned $500K in API usage in one month — $6M annualized — bingxu_ · 2026-11-10
- RTX 4090 Local Video Pipeline Turns Out a Clip in About an Hour — Distinct-Pie2389 · 2026-11-10