riderless: A Zero-Token Decision API on Gemma 4 Hits 93.1% Accuracy on One RTX 5090
Pale-Soil-2524 · reddit · 2026-09-22
The author open-sourced riderless (Apache-2.0), testing the hypothesis that a stock instruction-tuned model can serve as a decision engine if you skip generation and read label logits directly.
How it works
- Built on Gemma 4 26B-A4B: a FastAPI service over one llama.cpp child process
- Questions compile into prompts whose answer must be a single letter; prefill-only runs read the softmax over A-Z label logits at the final position
- Returns full probability distributions with zero generated tokens
Benchmarks on one RTX 5090 (Unsloth UD-Q4KXL GGUF)
- 0.931 accuracy on 1,286 questions across seven hand-written suites (extraction, claim support, record matching, guardrail gates, hierarchical classification, rerank, tool selection)
- 180 ms median sequential latency; opt-in batched mode reaches 108 ms at a cost of 5-6 answers
- Cross-question contamination probes: 0.0
Comparisons: Kev-9B scored 0.898, Laya's typed-decisions checkpoint 0.618, and so1 landed at 0.921 (separate) / 0.901 (packed) on the same suites—prompt/readout differences were worth only a point or two. Caveats: one run each, synthetic suites, no confidence intervals.
More from coding & agent
- Grounded Document Agent: cited PDF Q&A with LlamaParse, LlamaIndex and local Ollama — Roger_M_Taylor · 2026-09-22
- Synara v0.9.0 brings computer use to native macOS apps in beta — CurieuxExplorer · 2026-09-22
- scikit-learn co-founder, Bain AI lead to debate what agentic data science actually works — hugobowne · 2026-09-22
- Pi community ships 5 agent-team plugins as official sub-agents stay absent; Pi 0.87 splits session from context — solyarisoftware · 2026-09-22
- Pi v0.87.0 ships canonical session context editing, plus five breaking changes — solyarisoftware · 2026-09-22
- MiniMax details how to build a testbed for coding agent harness changes — MiniMax_AI · 2026-09-22