OpenJev: open-source Jev API clone on DiffusionGemma hits ~170ms latency, 198/201 accuracy
Every-Comment5473 · reddit · 2026-09-18
TypeSafe launched Jev on Tuesday — you feed it data plus typed questions (yes/no, pick-one, 0–N scale) and it returns a probability per option, absurdly fast. One developer, tired of the waitlist, built OpenJev, an open-source server exposing Jev's API, based on Matt Mastracci's vLLM PR #57250 that replicates the trick on DiffusionGemma with a single denoising step:
- Drop-in compatible: change the base URL and TypeSafe SDKs just work; Apache-2.0 code, a Docker image (vLLM + API in one container), and a free hosted API (100M tokens/account, prompts not stored).
- Latency: TypeSafe cites 3–329s for frontier LLMs on the same task; Jev runs 70–500ms; OpenJev's hosted endpoint hits 170ms p50 (73ms on GPU).
- Quality: Mastracci's live evals put DiffusionGemma-as-Jev at 198/201 vs Jev's 191/201, and faster on a DGX Spark.
It's v0.1 riding an unmerged vLLM PR, with the author's RTX PRO 6000 serving production.
Related event: OpenJev Open-Source Clone Runs Jev-Style API on One RTX 3090(3 posts)→
More from Infra
- SurrealDB 4 launches as multi-model ACID database for the agentic era — Scobleizer · 2026-11-04
- Fire at Taiwan Panasonic PCB materials plant may tighten CCL supply further — zephyr_z9 · 2026-11-04
- AMD CEO Lisa Su says revenue to grow over 80% in 2027, data center to top 80% of sales — AccBalanced · 2026-11-04
- Self-improving software is inevitable: closing the agent-observability loop — bibryam · 2026-11-04
- Gigabyte's AI TOP motherboard packs 2TB RAM and 9 GPUs for home AI labs — vincent_koc · 2026-11-04
- Building a 7-GPU 132GB VRAM local inference rig: consolidate on EPYC or split across two systems? — infectiousstupidity · 2026-11-04