Baseten's joey-1: Decision Models Answer in a Single Token, 10x Cheaper Than mammoth-1
baseten · x · 2026-10-30
Baseten's blog post details its third open-source decision model, joey-1 (2B). The core thesis: many LLM workloads don't need generated text, they need decisions — routing tickets, flagging messages, approving requests.
LLMs historically answer by generating text, adding cost and latency to every call, compounding across chained decisions. Decision models return the answer as a single token with a calibrated probability score instead. joey-1 serves the average 2,000-token request at a P50 end-to-end latency of 20ms (2.5x faster than elephant-1's 49ms), at $0.01 per million tokens — a tenth of mammoth-1's price. The Base Labs family now spans 27B/9B/2B, all accepting text and images; internal tests claim joey-1 is the strongest decision model at ≤3B parameters.
More from Infra
- PC downturn frees wafers for server CPUs, not accelerators, argues Bajarin — BenBajarin · 2026-10-30
- Amazon misses on sales and EPS as AWS grows 39.3% but falls short of buy-side bar — Kr00ney · 2026-10-30
- Hands-on: Magnitude claims 2x over llama.cpp, but disappoints on mid/low-end hardware — OsmanthusBloom · 2026-10-30
- Fujifilm to boost 100TB tape production 5x with $63M, its biggest tape bet in 20 years, on AI demand — ControlCAD · 2026-10-30
- Altman: Memory Makers Could Each Make ~$100B Annual FCF but Won't Build Fabs — beffjezos · 2026-10-30
- Top-10 on a GPU kernel leaderboard: why agent kernel search pays back in ~6 months — JensHonack · 2026-10-30