10 coding agents tested on local Qwen 3.8 27B: Pi wins on efficiency at 9.2k tokens per task
latent_lurker8587 · reddit · 2026-10-30
The author benchmarked 10 coding agents against a local Qwen 3.8 27B (Q4, llama.cpp) on a MacBook Pro M5 Max 64GB, in identical sandboxes with 5 trap-laden tasks.
Key findings
- Pi passed 5/5 with the lowest usage every task: 9.2k tokens, 38s median; Qwen Code was hungriest at 107.9k tokens/task
- Atomic Agent was the only one using llama.cpp's native endpoint with grammar-constrained tool calls, kept its prompt stable (86% cache reuse), stayed offline, and passed 5/5
- DeepSeek Harness had top cache reuse (83%); Hermes needs 64K+ context; Codex now requires llama-server's /v1/responses
- Codex, Cline, goose and OpenCode tried to reach the internet on every run despite telemetry off — the sandbox blocked it
- Three agents fell for a log-analysis trap by grepping "ERROR" (244 vs 64 real errors)
Includes setup notes on default telemetry (Cline, Qwen Code) and session-log uploads (DeepSeek Harness).
More from coding & agent
- Lean 4 formalization of an asymmetric metric-tensor physics paper hits GitHub — SubtleGradient · 2026-10-30
- YC-backed agent-native office suite launches free in ChatGPT and Codex — ycombinator · 2026-10-30
- Supabase never had an agent strategy — its CTO explains why it won anyway — socialwithaayan · 2026-10-30
- Understand Anything hits 88.9k stars: turn any codebase into a queryable knowledge graph — adnan_hashmi · 2026-10-30
- Scopewerk: Open-Source Rust App Puts a Hanmatek Scope on Your Desktop With MCP for AI Agents — debreuil · 2026-10-30
- The "did you" agent: a prompt for automated daily data QC — nlarusstone · 2026-10-30