Developer turns judgment model Jev into a chatbot: 3 tok/s and pricier than LLMs

_rchaves_ · x · 2026-09-18

Developer Rogerio Chaves gave Typesafe's judgment model Jev—which returns typed verdicts (Choice, Noul, Score) with probabilities instead of text—a caveman vocabulary and forced it to pick the next word like an LLM, creating CaveJev. His measured take:

Speed: Jev manages only 3 tok/s vs 150 tok/s for normal LLMs
Cost: with no caching of any kind, it would end up much more expensive
It also isn't RL'd for text generation

Still, he found it exciting: he can finally talk to the model and see how it "thinks," plus he now has a primitive to kind of rebuild an LLM as an API. The open-source jev-experiments repo includes latency/concurrency/accuracy benchmarks and a cost calculator.

Original post →

Developer turns judgment model Jev into a chatbot: 3 tok/s and pricier than LLMs

More from Fun