Lab open-sources 27B agent model: 15.1% hallucination rate, half the tokens of its base
lukas_m_ziegler · x · 2026-10-21
A lab open-sourced a 27B model built for one job: acting on unclear input without making things up.
- Hallucination rate 15.1% vs 30.4% for its own base and 85.3% for GPT-6 Luna
- Agent benchmarks: Terminal-Bench 2.1: 76.0 (best in its comparison set); τ³-bench: 259/278 (next best 220); MCPMark: 48/55 (next best 42)
- Cheaper outputs: 29.3K median output tokens per task vs the base model's 65.3K
Trade-offs: GPT-6 Luna still wins DeepSWE and the omniscience index; general-knowledge scores sit level with the base. This is targeted work on hallucination and token efficiency, not just a scaling run.
More from coding & agent
- Monid ranks agent tools by real usage: TinyFish leads with 754K weekly transactions — HeyAmit_ · 2026-10-23
- Claude cut a 2.6-hour video export to 20 minutes — but only after being told to rebuild the server — byhewar · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23
- Capy 0.5.0 lets Claude Pro/Max subscribers reuse their plan quota, adds OIDC cloud access — garrytan · 2026-10-23