Jev loses to Gemini on 1,565-email classification benchmark, but dev still wants it in production
socialwithaayan · x · 2026-09-20
A developer benchmarked TypeSafe's Jev model on 1,565 German and English business emails across 10 categories from industrial suppliers, and Jev lost to Gemini on email classification. Still, the tester remains interested in putting it into production — the interesting result wasn't accuracy but where the mistakes happened.
More from coding & agent
- Hindsight ships 0.8B model for agent memory decisions: 2.5x cheaper, 1.9x faster — victor_explore · 2026-11-04
- Human Oversight Is 70-75% of AI Agent Costs in Banking, McKinsey Finds — mikeflache · 2026-11-04
- Over 80% of Major Financial Institutions Now Run Agentic AI in Production — mikeflache · 2026-11-04
- Mnemos MCP nears launch after shared memory layer fix — RileyRalmuto · 2026-11-04
- Moving mobile dev to remote machines makes it multiplayer across product, design, engineering — rudrank · 2026-11-04
- Kuaishou's EvoRec: self-evolving agentic recommender lifts ad revenue 4.6% — _reachsumit · 2026-11-04