SoK of 139 Semantic Decision Engine Papers Finds Only 4 Back Real-Time Claims with Matched Measurement
Delong Li · hf · 2026-10-06
- Scope: Semantic decision engines (e.g., Jev) can return valid answers yet miss network deadlines, pick infeasible actions, or leave services unverified. The SoK systematizes 139 paper families by decision interface, execution path, and check ownership.
- Findings: Fifty families claim their engine fits a control loop or time budget, but only four support it with matched measurement; only four of 139 report deadline attainment. The gap concentrates where decisions lack a deterministic computation step: 72 families make 22 such claims, none supported, and only two name a coverage owner.
- Evidence: Bounded tests show each gap can reverse an admission verdict—a decision meeting a 10 s budget for every isolated request meets it for none once decisions queue; one engine passes one coverage check and fails another.
- Output: A minimum reporting record, design rules, and a research agenda for admitting decision engines into control loops.
More from Research
- AI Math Breakthroughs Prompt Question: How Many Real Engineering Problems Have Math Solutions? — Afinetheorem · 2026-10-06
- AI for Math Fund adds $17.1M from XTX Markets, backing 22 projects across 30 organizations — AlexKontorovich · 2026-10-06
- 62-person blind test of 48 LLM jokes shows models are getting funnier, Astra tops with 50%+ laughs — paraschopra · 2026-10-06
- Bayesian teaching dramatically improves probabilistic reasoning in LLMs, Nature Comms paper finds — tallinzen · 2026-10-06
- LiFT loops a DiT at inference: beats DiT-XL/2 with 52% less inference compute — cgmsnoek · 2026-10-06
- Researcher laments how slow humans look as algorithm design breakthroughs pile up — michaelchchoi · 2026-10-06