Jev runs at 143ms per question, hits 94% on sentiment with zero-shot prompts
colinmcnamara · x · 2026-09-21
Colin McNamara's hands-on run of TypeSafe's Jev: 143ms per question (157ms for ten), 94.0% on sentiment and 88.2% on news topics with no examples and no tuning. Notably, the vendor's own published failure cases reproduced exactly on his API key — the company documented where its model breaks, which is rare.
More from Models
- YOCO back in spotlight: blog breaks down cross-layer KV sharing in DeepSeek-V4.1-Flash and Gemma 4 — donglixp · 2026-09-21
- Gemini 4.0 Rumored Next: Yearly Pro Updates, Monthly Flash Releases Expected — haider1 · 2026-09-21
- SemiAnalysis' Dylan Patel: Chinese labs going closed-source, 'open is dying quickly' — ns123abc · 2026-09-21
- DeepSeek Web Output Style Reportedly Lobotomized by Safe Harbor RLHF — Old_Let6328 · 2026-09-21
- JEV opens to all with $5 free credits; $0.042 input and free output pricing — op7418 · 2026-09-21
- ChatGPT 20x Max users hit tighter limits, suspect compute favors government and API customers — BopSupreme · 2026-09-21