Jev: A Non-Autoregressive Decision Model That Outputs Structured JSON in 70ms
karminski3 · x · 2026-09-18
- Jev (by TypeSafe AI) abandons the autoregressive architecture: it can't output free text, but accepts a user-defined schema and emits discrete predictions with branch probabilities in valid JSON — via a special parallel sampling design that needs only one forward pass.
- Speed and price: as fast as 70ms end-to-end; input costs $0.042 per million tokens, output is free. Fit for high-frequency production decisions like flagging dangerous shell commands at bastion hosts, spam classification, or market-watching.
- Caveats: no local nodes means 120ms trans-Pacific cable latency (200ms floor in practice); the team calls it a "System One" model meant to be paired with a stronger System Two model for strategic judgment. Founder Diogo Almeida previously contributed to OpenAI's InstructGPT and early RLHF work. Signup is gated, but it's already callable via Vercel AI Gateway.
More from coding & agent
- ty to ship full support for recursive implicit type aliases, built on mu types — charliermarsh · 2026-09-19
- Converting video to 120fps with a single prompt via Runway MCP — tlakomy · 2026-09-19
- Experimental Jev-powered RLS linter catches Supabase policy bugs rule-based lint can't — SeanOliver · 2026-09-19
- Virtual green screen from a RealSense D436: depth-based RGB filtering, no backdrop needed — chrismatthieu · 2026-09-19
- Runway MCP turns video to 120fps with a single prompt — tlakomy · 2026-09-19
- Cost breakdown: 100 LLM calls in 8.5 seconds for roughly a quarter of a cent — narphorium · 2026-09-19