Open-source ninfer runs Jev-style decisions on Qwen3.8 27B at 84.4% accuracy
Unlucky-Message8866 · reddit · 2026-09-21
A Redditor built a Jev-like native decision API on Qwen3.8 27B with their open-source ninfer framework. On JevBench v1.2's public set (231 typed decisions) it scores 84.4% accuracy (vs 86.6% for official Jev 1.13.0), with excellent calibration (ECE 0.045, Brier 0.081) and p50 latency of 127ms on an RTX 5090. Easy tier hits 100%, hard tier 69.4%; weakest families are temporal-numeric (33%) and probability (50%). Self-described dirty POC with known allocation bugs; weights on HuggingFace.
More from Models
- ChatGPT now links to brand homepages far more, reshaping brand-mention tracking — lilyraynyc · 2026-11-12
- YC-backed Monte launches post-training platform to build proprietary models from your agents' real work — ycombinator · 2026-11-12
- Leak Claims OpenAI Will Launch Entire GPT-6.2 Family Tomorrow to Counter Opus 5.5 and Fable 5.2 — heypearlai · 2026-11-12
- ARC-AGI-3 leaderboard: models score 90%+ but at $10K+ per task — haider1 · 2026-11-12
- LMArena Launches Agent Arena for Head-to-Head Agent Evaluation — arena · 2026-11-12
- LLMs lack hyperlocal municipal data, explains disputed Texas power outage debate — csuwildcat · 2026-11-12