Jev-compatible self-hostable API lands fast, runs 64 parallel tasks in under 1s with SGLang
iamrobotbear · x · 2026-09-18
ekzhang1 launched a Jev-compatible public API inspired by @typesafeai:
- Runs the comparable open model Qwen3.6-35B-A3B
- Uses SGLang radix cache to preserve prefill reuse, enabling very fast parallel system-one generation — 64 tasks in under 1 second
Boosters compare the buzz to the DeepSeek drop, and note strong demand for private/local deployments: half of their ideas would require the "system one" model to be private, and half of those local.
Related event: Developer ships Jev-compatible self-hosted API on Qwen3.6(2 posts)→
More from Infra
- Big Tech's Future Data Center Leases Top $1.3T, 70% Added in Last Four Quarters — rohanpaul_ai · 2026-11-02
- Daniel Lemire: AI is exploding demand for boring compute, server CPUs to double — lemire · 2026-11-02
- Physical AI's real story: a 4-layer map of 15 companies behind humanoid robots — pdamodaran · 2026-11-02
- Under 5% of US consumers pay for AI, yet compute demand is already forcing usage cuts — NinaDSchick · 2026-11-02
- Sam Altman: AI query energy cost fell 1,000x-100,000x in 3-4 years — rohanpaul_ai · 2026-11-02
- Marvell and Lumentum to merge, targeting $550B AI connectivity market — BenBajarin · 2026-11-02