Jev after days of real testing: weak Chinese, no reasoning, 30x faster than DeepSeek only in closed tasks
ZeYanjie · x · 2026-09-20
After days of heavy use, the author maps Jev's real boundaries: it only works with strictly defined constraints, not open-ended tasks.
- Weak Chinese: official docs flag low CJK accuracy; the author found Chinese results erratic, English short texts stable.
- No reasoning: in Browser Use founder's long-horizon browser test Jev scored 1/20 vs reasoning-capable Luna's 17/20.
- Fails real webpages: polished demos run in clean sandboxes; messy DOM, async loading, and popups break it.
- 'Zero hallucination' ≠ correct: it only guarantees schema-valid output; the CEO confirmed it can confidently pick wrong options.
- Can't do dynamic tool-call parameters (line numbers, generated code, custom queries).
- No rationale output: only final choices/numbers, unusable for audits or human review.
Verdict: Jev shines in narrow, closed, latency-sensitive jobs — a mail-triage demo sorted 300 emails into 15 departments in 9.9s for $0.036, while DeepSeek read only 10 emails in the same time. The author calls for a community-built Jev harness.
Related event: Hands-on Tests Reveal Jev's Limits: Weak Chinese, Narrow Use(2 posts)→
More from Models
- Gemini 4 Carbon opens to all at $2/$10, price doubling to $4/$20 on Dec 1 — socialwithaayan · 2026-11-12
- Ten wild Gemini 4 Carbon demos in 24 hours: CUDA meadows, V8 engines, mech stomps — socialwithaayan · 2026-11-12
- Google quietly launches Gemini-decision-1 model and Decision API, leak claims — gaganghotra_ · 2026-11-12
- How to cut Fable 5.2 token costs by 90%: Arc routing plus a lean CLAUDE.md — EXM7777 · 2026-11-12
- Grok Bot 1.0 launches with support for joining Meet, Zoom and Teams calls — mark_k · 2026-11-12
- Partial RuneBench results: Inkling 2 Small scores 5.314, far above original — mariofilhoml · 2026-11-12