notjev turns any local LLM into a 23ms decision engine with one token and logprobs
n8tz · reddit · 2026-09-22
A developer released notjev, a small Node library that turns any OpenAI-compatible model into a fast System 1 decision engine: instead of generating text, the model outputs a single letter token (A/B/C...) with logprobs enabled, keeps only the option-letter probabilities, renormalises them, and returns a verdict, probability, margin, and a tunable abstention threshold.
Key points:
- Zero runtime dependencies; works with vLLM, llama.cpp, Ollama, OpenAI — anything supporting logprobs
- Uses the same weights as your chat model for a true System 1 / System 2 split: thin margins escalate to the same model in normal chat mode
- Speaks the Jev /v1/systemone wire contract
- Fast: 23 ms p50 on Qwen3-8B with llama-server
Repo: github.com/9pings/notjev
More from coding & agent
- Developer uses Codex computer use to navigate Austria's messy FinanzOnline tax portal — kevinkern · 2026-09-22
- Multi-agent group chats: drop a task at night, run 10+ agents unattended, review by morning — huangyun_122 · 2026-09-22
- 7 open-source repos for scraping millions of web pages, from Scrapling to ScrapeGraphAI — JafarNajafov · 2026-09-22
- If agents can shop for you, agent-native rivals will replace Zomato, Amazon and Blinkit — vaibhavbetter · 2026-09-22
- Refunds, fraud signals and high-stakes calls: where should agent autonomy stop? — ConvertMyStore · 2026-09-22
- Obsidian Starter Kit v4 turns your vault into an AI workspace with 375 skills and MCP server — dSebastien · 2026-09-22