llama.cpp Adds Decision Models: Scored Options in One Forward Pass via /v1/systemone
adnan_hashmi · x · 2026-10-03
A Hugging Face community article introduces decision model support in llama.cpp server via the new /v1/systemone endpoint: send a state (text, JSON, or a screenshot) plus typed questions, and the model returns a probability for each option in a single forward pass, following the System One format introduced with TypeSafe's Jev model—existing clients only need a new base URL.
- How it works: chat models spend one forward pass per token and need output parsing; decision models read the input once and always answer with one of your options plus a probability.
- Typical uses: request routing, content moderation, verifying agent steps, choosing an agent's next action.
- Supported models: Julia-1 (144M, 3ms), Laya (421M, 5ms), Kev-4B (4B, 12ms), lev (4B, 36ms), and OpenJev (27B, 43ms, multilingual + image support), built on mmBERT and Qwen3.5 bases.
Quick start: llama serve -hf ggml-org/Kev-4B-GGUF. Models are in the Decision models collection, with a community Decision Index for comparisons.
Related event: llama.cpp adds Decision Models: one forward pass scores options directly(8 posts)→
More from coding & agent
- AI code is cheap; undoing its confidently repeated mistakes is not — sujingshen · 2026-10-03
- Orchestrator v0.13.3 ships kanban-style management for multiple coding agents — julianweisser · 2026-10-03
- He let a computer-use agent run his Tinder: reading profiles and swiping right by checklist — cneuralnetwork · 2026-10-03
- Pavan Belagatti releases complete hands-on guide to the agentic software factory — Pavan_Belagatti · 2026-10-03
- Agent workloads move infra bottlenecks beyond model latency, engineer explains — AccBalanced · 2026-10-03
- Personal Agent Demos Are Stuck on Restaurant Bookings While Users Want Taxes and Invoices Handled — sujingshen · 2026-10-03