14B judge no better than 8B: one prompt tweak lifted grading from 3/14 to 10/14
Lost-Masterpiece2400 · reddit · 2026-10-17
A developer running a fully local agent setup (9070 XT + 3060 via Ollama) shares a counterintuitive finding: structure matters more than model size.
- Setup: a coder model fixes broken services; a separate judge model grades its diagnosis
- Tested Qwen3 14B vs qwen3:8b as judge — the 14B did no better and ran 3x slower because it spilled out of the 3060's VRAM
- Root cause: the judge was marking the coder down for commands the coder had only proposed but never ran
- Adding one note to the judge prompt explaining that distinction lifted correct grades from 3/14 to 10/14
Small sample, not proven — but a useful lesson for anyone building agent evals: fix the rubric before upgrading the model.
More from coding & agent
- Poll: 70% say chat is not the final form of agent UI — jonathan_wilke · 2026-10-17
- 'Yes, I'm a Full Stack Engineer': The Joke About Juggling Claude, Codex and OpenCode Models — rachittshah · 2026-10-17
- Reddit dev shares checkpoint-and-clear loop for running AI agents for days without losing state — Motiv31 · 2026-10-17
- Building a notes chatbot: why semantic search fails on dates and membership queries — bzarnal · 2026-10-17
- Reddit user slams Codex for over-engineering: turning a 30-second task into an hour-long run — Reaper_1492 · 2026-10-17
- 10 open-source repos for training small models to do your agent's job — victor_explore · 2026-10-17