Judgment-only model Jev opens to all with $5 free credit, sparking calibration debates
机器之心 · wechat · 2026-09-21
TypeSafeAI has removed Jev's waitlist, opening the structured-judgment "SystemOne model" to everyone with $5 (120M tokens) of free credit. Developers feed it a state plus a question with fixed options, and it returns a choice, a probability, or an ordered score that code can consume directly — suited for model routing, moderation triage, and agent tool-call risk checks.
Praised generalization: Sebastian Raschka notes Jev accepts natural-language labels at runtime, avoiding per-label-set retraining that plagues traditional encoders; he suspects the secret is data and API design rather than the training algorithm. An unconfirmed report says it was post-trained on fully synthetic data atop an existing model.
Reliability questions: users report inconsistent probabilities across identical prompts and heavy order sensitivity; @predictaddict found Jev's calibration trails CatBoost across eight real datasets. TypeSafeAI itself flags instability on arithmetic, counting, and adversarial inputs. One user's Jev-powered trading bot has lost $31,680.
More from coding & agent
- Plane Agents launch: assign work, tag in comments, or run on a schedule — JosephJacks_ · 2026-09-22
- Latent Space pod: TypeSafe CEO argues AI should make reliable decisions in software, not chat — lennysan · 2026-09-22
- Same model bills 5x more in a different harness: 21 combos tested across 60 tasks — CShorten30 · 2026-09-22
- Grok bot drives local uncensored Flux2 via ComfyUI MCP for horror art — PurzBeats · 2026-09-22
- Jev + Treg combo powers 3 GTM workflows, saving $8k/month in automation costs — iamrobotbear · 2026-09-22
- Trace: a terminal tool that feeds AI only the evidence it needs to debug — jse78 · 2026-09-22