Open 27B model on a single GPU matches Jev's accuracy with better calibration
colinmcnamara · x · 2026-09-21
Colin McNamara ran the same human-labeled items through an open 27B model on his own GPU. Accuracy was within 1.4 points of Jev; the open model had better calibration on sentiment (0.04 vs 0.09 ECE); Jev was 1.8–4.5x faster. One model, one machine, two well-known datasets — a first look.
Related event: Jev tested: prompt tips and an open 27B model matches it(2 posts)→
More from Models
- MiMo near-SOTA on DeepSWE with just ~$2.6M RL run: will data cost more than training? — my_cat_can_code · 2026-09-21
- humansand's Persimmon model learns to share info gradually like humans, with Trickle Test — niloofar_mire · 2026-09-21
- Why yes/no answers are fast for LLMs: output tokens dominate latency — tinyfool · 2026-09-21
- ChatGPT reportedly removes free-tier chat limits, offering unlimited text chats — nikola_mr64990 · 2026-09-21
- Users say top-tier Astra is too costly, hope GPT-6 fixes token economics — CtrlAltDwayne · 2026-09-21
- TypeSafe's JEV model fully open with $5 free credit, powers 1-second 3D scene generation — tinyfool · 2026-09-21