llama.cpp Adds Decision Models: Scored Options in One Forward Pass via /v1/systemone
adnan_hashmi · x · 2026-10-03
A Hugging Face community article introduces decision model support in llama.cpp server via the new /v1/systemone endpoint: send a state (text, JSON, or a screenshot) plus typed questions, and the model returns a probability for each option in a single forward pass, following the System One format introduced with TypeSafe's Jev model—existing clients only need a new base URL.
- How it works: chat models spend one forward pass per token and need output parsing; decision models read the input once and always answer with one of your options plus a probability.
- Typical uses: request routing, content moderation, verifying agent steps, choosing an agent's next action.
- Supported models: Julia-1 (144M, 3ms), Laya (421M, 5ms), Kev-4B (4B, 12ms), lev (4B, 36ms), and OpenJev (27B, 43ms, multilingual + image support), built on mmBERT and Qwen3.5 bases.
Quick start: llama serve -hf ggml-org/Kev-4B-GGUF. Models are in the Decision models collection, with a community Decision Index for comparisons.
More from coding & agent
- Devs flag GPT's defensive programming for hiding bugs; OpenAI asks for /feedback cases — pvncher · 2026-11-19
- Flutter ships A2UI 1.0: agents respond with real interactive widgets, not chat text — rseroter · 2026-11-19
- Skills vs MCP vs RAG vs Memory: the four context mechanisms AI agents actually need — MaryamMiradi · 2026-11-19
- Anthropic details agent memory: 'dreaming' lets Claude organize its experience — daniel_mac8 · 2026-11-19
- The case for externalized memory as the real fix for continual learning — daniel_mac8 · 2026-11-19
- FT: US tech outages blamed on rush to fix bugs found by AI — nordicinst · 2026-11-19