System 1 models return typed answers in one encoder pass: 45ms per query on a laptop CPU, but often confidently wrong

ghumare64 · x · 2026-09-26

A hands-on review of a new class of "System 1" models that take text plus questions with closed answer sets and return a probability distribution from a single encoder pass — no generation, tens of milliseconds per call, a tiny fraction of LLM cost, and results your code can branch on directly.

Laya is the open version (Apache-2.0 weights, by Nandakishor M of Convai Innovations, on GitHub/PyPI, ships a server speaking Jev's wire format); Jev is the hosted closed version from TypeSafe AI (TechCrunch, Sept 18).
Author ran Laya 0.3.20's smallest checkpoint on a laptop CPU: 45ms per question.
On 30 hand-labeled support tickets it scored 70%, and 6 of its 9 errors came at 0.98+ confidence — e.g. classifying "I do not want to cancel my account" as cancelaccount. Probabilities are only as good as your labeled data.
The piece also walks through request shape, how the probability vector is produced, how to pick a confidence threshold, and the economics of routing 10k tickets a day off an LLM.

Original post →

System 1 models return typed answers in one encoder pass: 45ms per query on a laptop CPU, but often confidently wrong

More from coding & agent