Turn any local LLM into a confidence-scored classifier via logprobs, full llama.cpp recipe included
DivideHorror3217 · reddit · 2026-09-21
A Reddit user shares a practical trick to use any GGUF model as a confidence-scored classifier (JEV-style) with llama.cpp, no classifier training needed.
Key steps:
- Launch llama-server with npredict=1 and nprobs=10, reasoning disabled
- Prompt with strict instructions (e.g. spam: reply only 1 or 0)
- Set maxtokens=1 with logprobs enabled, then exponentiate the top logprobs to get probabilities
In their test, "1" scored logprob -0.0046 (99.5% confidence) on a spam sample while "0" got 0.5%. On a 4GB VRAM GPU with 170GB/s bandwidth the full run took 634ms; they estimate 30-75ms on an H200.
More from coding & agent
- Z.ai Open-Sources Its Coding Agent Workbench ZCode Under Apache 2.0 — cephaloform · 2026-09-21
- A new role emerges: 'vibecoder relations' as non-coders drive model launches — venturetwins · 2026-09-21
- 220K agent tool calls analyzed: model-watching-model Jev great at progress tracking, weak at security — hrishioa · 2026-09-21
- service_tier=fast rejected on ChatGPT subscription, API-only parameter confirmed — TrickyPlastic · 2026-09-21
- Powermove launches: a tiny-kernel video editor where every feature is an AI-writable extension — round · 2026-09-21
- Using closed-decision Jev-like models to kill tag hallucination in dataset captioning — Iory1998 · 2026-09-21