Hands-on with Laya, an open System 1 model: one encoder pass returns typed probabilities in milliseconds
ghumare64 · x · 2026-09-26
A 19-minute deep-dive and hands-on test of the new "System 1" model class from TypeSafe AI. These models generate nothing: you send a state (a ticket, JSON record, transcript) plus questions with closed answer sets, and a single encoder pass returns a probability for every answer — routing or yes/no calls take tens of milliseconds at a fraction of LLM cost, returning a type your code can branch on.
- Jev: hosted/closed, from TypeSafe AI, previously covered by TechCrunch. Laya: open Apache-2.0 weights on GitHub/PyPI, shipping a server speaking Jev's wire format.
- The author ran Laya 0.3.20's smallest checkpoint on a laptop CPU, carefully labeling which numbers are self-measured vs README-reported.
- One support ticket is traced through both systems: request shape, probability generation, and cost in milliseconds.
- Also covered: where the probabilities lie, how to pick a confidence threshold, and whether to move real ticket volume off an LLM.
- Caveat: probabilities are only worth what your labeled data says; the open checkpoint was "often confidently wrong" in his tests.
Related event: System 1 models classify tickets in milliseconds on laptop CPUs(3 posts)→
More from Models
- One prompt gets Claude Opus to build a third-person League of Legends action game — TAbrodi · 2026-09-26
- Claude's Sweet Music Videos Creep Out a Self-described Non-doomer — ctjlewis · 2026-09-26
- Unified Cowork/Projects View Across Machines Has One User Saying Anthropic Is Back — toptickcrypto · 2026-09-26
- Open System 1 model runs at 45ms per query on laptop CPU, 70% on support tickets — ghumare64 · 2026-09-26
- Researcher says Claude solved his months-old multimode spectrum problem in hours — jwt0625 · 2026-09-26
- ARC Prize once insisted plain deep learning couldn't crack ARC — then o3 happened — teortaxesTex · 2026-09-26