Baseten open-sources joey-1, a 2B decision model with 20ms latency at $0.01 per million tokens
Base Labs, part of Baseten, open-sourced its third "decision model" joey-1 (2B parameters) on October 30, targeting extremely low latency and cost: the team reports a P50 end-to-end latency of 20 milliseconds for 2000-token requests—2.5x faster than the previous elephant-1—at $0.01 per million tokens, roughly one-tenth of traditional approaches. This is the team's third open-source decision model, after mammoth-1 (27B) and elephant-1 (9B).
Confirmed
- Positioned for "System 1 decisions": given structured context and a question, a single forward pass returns a one-token answer plus a calibrated probability score, with no free-text generation and parsing needed
- Use cases include request routing, ticket triage/escalation, message flagging, approval decisions, and other classification or yes/no judgment tasks
- Performance and pricing: P50 end-to-end latency of 20 ms (2000-token requests), $0.01 per million tokens
- The official blog lays out the rationale for the "decision model" category: many LLM use cases need decisions rather than text generation, and the traditional generate-then-parse approach stacks cost and latency onto every call
Why it matters
- By compressing "generate then parse" into a single-token output, decision models can substantially cut cost and latency in high-volume, high-frequency scenarios (routing, moderation, classification), representing a differentiated branch of the LLM application paradigm
- The consecutive open-sourcing of three models at different scales (27B/9B/2B) shows Base Labs systematically building out "decision models" as a standalone category
2026-10-30 ~ 2026-10-30 · 5 related posts
Primary sources
- [source] Baseten releases joey-1, a 2B open-source decision model with single-forward-pass probabilities — baseten · 2026-10-30
- [source] Baseten's joey-1 hits 20ms P50 latency at $0.01 per million tokens — baseten · 2026-10-30