llama.cpp adds /v1/systemone endpoint to run decision models locally, returning probabilities
solyarisoftware · x · 2026-10-03
A new llama.cpp update lets you run decision models locally: give it a question and choices, get probabilities back—no generated answer. The new /v1/systemone endpoint targets request routing, tool selection, and checking an agent's work.
Supported GGUFs: Julia-1 (144M), Laya, lev, Kev-4B, and OpenJev (27B, with image support). OpenJev's weights are CC BY-NC 4.0; the other four are Apache 2.0. Update llama.cpp to try it.
More from Infra
- AMD Loom boosts Qwen3.5-4B prefill up to 2.6x on Radeon 890M laptops — jfowers_amd · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS Lambda Adds GPU Support via Managed Instances for Inference and Media Workloads — DavidWells · 2026-11-19
- Nvidia Allegedly Made GPUs Artificially Scarce to Keep Prices Sky-High — GaryMarcus · 2026-11-19
- AWS adds private CA support to Bedrock AgentCore Identity for connecting agents to self-hosted IdPs — AWS ML Blog · 2026-11-19
- Leak Hints at Classified 1M-GPU Training Cluster in China That Skips HBM — teortaxesTex · 2026-11-19