Jev Router latency deep-dive: 6.18s median vs 3.64s for DeepSeek, P90 gap widens to 30.6s
arena · x · 2026-10-08
A follow-up from Agent Arena's Jev Router evaluation comparing end-to-end request latency against nearby Pareto-optimal models:
- Jev Router's median latency was 6.18 seconds vs 3.64 seconds for DeepSeek V4.1 Flash (Max).
- The P90 gap widens further: 30.59 seconds for Jev Router vs 14.26 seconds for DeepSeek.
- Consistent with the main evaluation: the router loses on both cost and latency, with its value concentrated in steerability.
More from Models
- Skeptical of 'small model for evals' moats: labs will distill it into cheaper models — Shahules786 · 2026-10-08
- Bengio disputes 'just a sandbox bug' framing of AI agent hacks in FT op-ed — AlexTensor · 2026-10-08
- OpenAI model proves Hilbert's Tenth Problem false over Q, sidestepping 80-year approach — aran_nayebi · 2026-10-08
- What Anthropic's $200 tier changes about choosing between Opus and Sonnet — thursdai_pod · 2026-10-08
- AI flip: it may plan your Boston trip before solving the Riemann hypothesis — jxmnop · 2026-10-08
- OpenRouter's Usage and Spend Charts Have Split: Used Models Aren't the Paid Ones — Spiritual_Skirt_9312 · 2026-10-08