421M-parameter open-source Laya returns answer probabilities in one pass at 40ms on a T4
maier_ak · x · 2026-09-28
Laya is a 421M-parameter open-source model (Apache 2.0) built by a solo developer: it takes text plus typed questions and returns answers with probabilities in a single forward pass—no text generation at all—running at 40 ms on one T4 GPU.
Its closed counterpart, Jev from TypeSafe AI, is API-only at $0.042 per million input tokens. Independent benchmarks put Jev at 264-276 ms median latency, 0.910 on AG News, and a calibration error of 0.246—the worst in that study.
More from Models
- Uncensored Qwen-based 27B OrcaSAQ-2 GGUF trends on Hugging Face — orcarouter · 2026-09-28
- Scoop: No Kimi 3.1 launch next week either, says insider — ChrisGPT · 2026-09-28
- AI Circle Mocks Model IQ Rankings: 'Just a Vocab Quiz Leaderboard' — repligate · 2026-09-28
- repligate: Sonnet 3's Values and Inner Ontology Are Strikingly Consistent Across All Instances — repligate · 2026-09-28
- Premier model output now runs $4-6 per Mtok at labs with always-on bot products — willcb · 2026-09-28
- Hobbyist width-prunes a 30B model to 14B and distills it, passing 57 of 60 agentic tool tasks, Apache-2.0 — ZenZombie117 · 2026-09-28