Ex-Meta engineer benchmarks Jev vs GPT-nano: same score, 5x faster, 20% pricier
danielmckinn0n · x · 2026-09-25
Former Meta engineer Daniel McKinnon shared hands-on benchmarks of Jev, questioning the hype: his first LLM project in 2022 used OPT-175B as a first-line classifier, so Jev felt like "just LLM + constrained decoding."
With few public benchmarks available, he compared Jev against GPT-nano on a large-scale document classification task:
- Identical scores on the benchmark
- Jev was 5x faster
- But 20% more expensive
He finds the speed tradeoff worthwhile for his use case, but suspects a fine-tuned small model like Qwen or Gemma could be faster, better and cheaper for continuous workloads — and openly asks for an ELI5 on what the excitement is about.
More from coding & agent
- User ditches GUI: local LLM now his main PC interface for installs, GRUB and more — Forward_Jackfruit813 · 2026-09-25
- Where an agent loop runs, how it fails, and how to build durable sessions — tempNull · 2026-09-25
- Community turns the Opus 5.5 pixel-art prompt into an open-source pixel-anims skill — majidmanzarpour · 2026-09-25
- Opus 5.5 + one prompt generates animated pixel art wizard purely in code — majidmanzarpour · 2026-09-25
- Claude offers $250 cloud credit, migrates Claude Code sessions to the cloud — MicahBerkley · 2026-09-25
- A visual harness for Claude Code: watching your agents work after hours — EricBuess · 2026-09-25