Cascading decision engine: 2ms Markov chains, 4B MoE fallback, 44ms median

inawhole · reddit · 2026-10-15

The open-source Gevva0 project presents a multi-tier cascading decision engine (figures are the author's claims, not independently verified):

• Tier -1 static rules (<50µs): hot-reloadable regex/keyword guards with no GPU cost
• Tier 0 absorbing Markov chains (<3ms): EmbeddingGemma-2 (740M) embeds queries into a 768D manifold and solves random-walk absorption probabilities in closed form — 100% early-exit accuracy with zero false exits on 344 out-of-domain items
• Tier 1 fast-path logits (35ms): a single forward pass on Gemma 4 26B-A4B MoE (4B active) with a custom K=64 micro-head slicing the 262,144-row vocabulary to just 52 answer letters, cutting output-head compute 99.98% and resolving 77.9% of queries without generating a token
• Tier 2 hot-KV bounded CoT (≤256 tokens): escalates the hardest 17.1% of requests directly from the KV cache Tier 1 prefilled, eliminating double prompt processing and cutting long-context latency up to 30%

Claimed results: on the author's JevBench, Gevva0 beats Claude Opus 4.6 Thinking on chance-corrected intelligence (84.62 vs 83.90) at 44ms median latency (vs 4,850ms); the 4B-active cascade reportedly outperforms dense 27B models at 60ms p50 (7x faster). A Pacman demo shows 60 FPS real-time game-loop readiness.

Repo: https://github.com/solvingSteve/Gevva0

Original post →

More from Infra

• OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
• Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
• ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
• MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
• "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23
• Public data is drying up; private training data is turning into a browsable catalog — SucceededMind · 2026-10-23

Infra channel →