A local 9B assistant went from 6/32 to near-perfect: most fixes were code, not prompts

Macvedus · reddit · 2026-11-14

The team built Atlas, a personal assistant running locally: an open-weight model in Ollama, calendar over CalDAV, tasks in a Markdown file, a Tailscale-served phone page, and morning briefings via web push. They benchmarked 32 requests ("What's on my calendar Thursday?", "Move my dentist appointment…") in fresh conversations.

First score: 6/32 — a spec-built qwen3.5:9b with no tool instructions called tools twice, asked which year "Thursday" was in, claimed events it never created
The program asks, not the model: every mutating tool ships with a second function that generates the confirmation question ("Move Dentist from Thu 16:00 to Fri 10:00?"), re-confirmed before Yes runs anything
Check the reply, not only the request: per-tool instructions took 9B from 15 to 22; the rest was code — title-keyword event lookup, one-day hints, defaults for optional args, reminders when the reply announces a tool instead of calling it. Final: 31/32 across three 9B runs, 32/32 with qwen3.8:27b, 27–28 with the 4B
Small models skip optional args: fixed with Python defaults

From their book Build Your Own AI Butler (Weekend AI Projects Vol. 1); free sampler linked in comments.

Original post →

A local 9B assistant went from 6/32 to near-perfect: most fixes were code, not prompts

More from coding & agent