Absorbing Markov Chains beat naive cosine routing for <2ms zero-token intent triage
inawhole · reddit · 2026-10-17
A dev replaces naive cosine routing with Absorbing Markov Chains over a 768D embedding space: instead of being baited by surface keywords ("Checkout payment webhook failed" → misrouted to Billing), the query is projected onto a 60-node domain manifold and B=(I-Q)^-1·R is solved on GPU in <0.05ms, absorbing into the correct domain before waking the LLM.
JevBench cascade results:
- Tier 0 (AMC + EmbeddingGemma2): avg 22ms, 11/11, 100% precision
- Tier 1 (Gemma4 26B A4B): avg 42ms, 91.49%, handles 78% of items
- Tier 2 (Gemma4 + CoT): avg 1,560ms, 75.61%, handles 17%
- 4-tier cascade: 46ms p50, 89.17% overall accuracy
Tier 0 drops to 1.5ms with cached static candidates. Three tripwire guards (negation/syntax, manifold proximity OOD check, entropy & margin gate) catch low-confidence routes. Useful for support ticket routing, agent tool calls, and classification — most queries never consume LLM tokens.
More from coding & agent
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23
- Capy 0.5.0 lets Claude Pro/Max subscribers reuse their plan quota, adds OIDC cloud access — garrytan · 2026-10-23
- You.com CEO Richard Socher: legacy code is human slop, let AI clean it up — RichardSocher · 2026-10-23
- Let agents design a single-line glance component so you can track them from the sidebar — blixt · 2026-10-23