Local GGUF Coding-Agent Benchmark: 35B Matches 125B, Tool-Calling Is the Real Lever

Joralk_ · reddit · 2026-10-22

The author built the ggufbench harness and benchmarked seven local GGUF models plus a Strata engine on 2× RTX 5080 for agentic coding rather than chat: can the model call tools with well-formed arguments, survive a multi-call loop, read a folder, edit a file and verify its own change.

Method: 18 tool-calling, 72 code and 72 agent tasks per model, three seeds each (42/43/44), identical sampling (temp 0.2, top-p 0.95, top-k 40, min-p 0.05, KV q80, one slot), each model in its own best profile, every exclusion counted with a reason.

Results (thinking off): MiniCPM-V-4.7-35B-A3B Q4KM leads at 0.85 global (tools 100% / code 83% / agent 90%, 26ms); qwen36-35b-distill 0.84; Qwen3.8-27B variants 0.81–0.82; gemma-4-26B 0.80; the 125B Qwen3.8-Flash-Next on Strata only reaches 0.80 and wins nothing.

Takeaways:

Size is not the lever for agentic coding; tool-calling reliability is, and the 27B class already saturates it
Thinking mode is a tax that only pays on hard code: the 27B goes 0.79→0.93 on code and 0.90→0.96 on agent, at 36ms→629ms and +253 median tokens per call; off by default, on for the hard tier
Swapping engines (BigMoeOnEdge) made no measurable difference (0.920 vs 0.923) but surfaced a router-hook bug returning empty messages, filed as an issue

Limitations: consumer GPUs, self-written task set, one machine, one quant per model.

Original post →

Local GGUF Coding-Agent Benchmark: 35B Matches 125B, Tool-Calling Is the Real Lever

More from coding & agent