Burkov Dismisses Mystery Project Jev as Likely BS

After receiving multiple email inquiries, Andriy Burkov, author of The Hundred-Page Machine Learning Book, publicly assessed the mysterious project Jev on September 21, bluntly calling it likely BS; the post spread widely in the ML community, and he added two technical criticisms on September 23.

Confirmed

Jev claims to be much faster than LLMs on the same tasks, but its benchmark is regular LLMs rather than diffusion LLMs; against diffusion LLMs the advantage may vanish, and Jev itself might even be a diffusion LLM.
Burkov argued the speed advantage likely matters only in rare scenarios such as high-frequency trading.
On September 23 he raised two technical points: first, the time before an LLM emits its first token equals the compute spent on its decision, so if Jev's full-output time matches a same-size LLM, speed and quality cannot both hold — and he offered an equal-speed LLM alternative; second, so-called calibration can align softmax probabilities with ground-truth probabilities in training data via RL, but LLM probability calibration on open-ended questions is unsolvable, and calibration does not equal true probability.

Why it matters

Burkov's critique targets a common AI marketing tactic: choosing a favorable baseline (regular LLMs) instead of the strongest rival (diffusion LLMs) to inflate speed claims — a reusable review framework.
The two arguments — first-token latency as decision compute, and calibration not equaling true probability — give the community concrete lenses for weighing speed-quality tradeoffs and probability reliability.

2026-09-21 ~ 2026-09-23 · 5 related posts

Primary sources

Burkov calls stealthy LLM-rival project Jev 'BS' over speed and calibration claims — burkov · 2026-09-21 03:51 (UTC+08:00)
Burkov: Jev can't beat LLM speed-quality tradeoff — here's the KV-cache trick that matches it — burkov · 2026-09-23 00:43 (UTC+08:00)
Andriy Burkov on why Jev-style LLM calibration can't beat an LLM at real probabilities — burkov · 2026-09-23 01:24 (UTC+08:00)

[source] Burkov calls stealthy LLM-rival project Jev 'BS' over speed and calibration claims — burkov · 2026-09-21
Burkov dismisses Jev's speed claims; suggests ONNX DeBERTa classifiers beat LLMs — MaziyarPanahi · 2026-09-21
Andriy Burkov calls Jev "BS": speed advantage may vanish vs diffusion LLMs — gethackteam · 2026-09-21
[source] Burkov: Jev can't beat LLM speed-quality tradeoff — here's the KV-cache trick that matches it — burkov · 2026-09-23
[source] Andriy Burkov on why Jev-style LLM calibration can't beat an LLM at real probabilities — burkov · 2026-09-23