Jev publishes own intelligence benchmark topping at 60.0, warns 'don't trust it'
_lewtun · x · 2026-10-15
TypeSafe/Jev released official benchmarks for its models: jev 1.14.0 scores 60.0 and jev 1.13.0 58.9 on an internal "Overall intelligence" index, ahead of open models kev-27b (46.7), ppplx-decider-v1.1-27b (46.0), clef (44.3), clef-flash (38.8), kev-4b (36.9) and d1-3b (33.8). The company admits the eval is likely biased toward its own models, suggests focusing on relative ranking of open models, and excluded hosted models over data and validation-set security concerns. multimodalart praises the honest disclaimer and plans to eval jev 1.14 & vision in his v0.4 Decision Index.
More from Models
- Applied Compute's Spork post-training recipe doubles chess puzzle solve-rate vs GRPO — ypatil125 · 2026-10-15
- Claude vs ChatGPT for Coding: Users Weigh Limits and Credits — OkraSuper9329 · 2026-10-15
- Google's Gemini 4 Argon Tops Physical-World Understanding, Scores 59.1% on Long Video — 141_1337 · 2026-10-15
- Model greets evaluators in its chain of thought — 4 trends paint an alarming AI trajectory — dfrsrchtwts · 2026-10-15
- TypeSafe AI ships Jev 1.14.0 with experimental images hidden in danger-zone flags — michellechen · 2026-10-15
- ADL says it works with OpenAI, Anthropic, Google, Meta and xAI to curb antisemitism; Claude translation blocked by safety filter — amasad · 2026-10-15