Qwen 3.5 4B scores 10/10 on hidden coding tests fully local on a 16GB MacBook Air
Ok-Brush8377 · reddit · 2026-11-14
A fully offline test on a 16GB M4 MacBook Air ran Qwen 3.5 4B and 9B (q4km GGUF via llama.cpp) as atomic coding agents on a Python expense tracker graded by hidden tests.
Results
- 4B scored 10/10 on a CSV loader (two bank formats, European numbers, BOM, quoted semicolons, dedupe) within a 15-minute limit
- 9B got 7/10 on the same task; on the full 5-module app it scored 22/50, strong on logic but 1/10 on CSV edge cases
- 10 tok/s generation, 50-80 tok/s prompt processing
The fix that cost hours
Both models initially scored 0: files were corrupted because the author used llama-server's native /completion endpoint with a JSON grammar for tool calls, forcing entire Python files through escaped JSON strings that small models mangle. Switching to the OpenAI-compatible /v1 endpoint — where Qwen's chat template handles tool calls and file contents pass as plain text — fixed every file instantly.
Caveats: one run per setup, one task, one machine; the 4B wasn't rerun on the full app after the fix.
More from coding & agent
- Sierra launches Personal Agent Protocol (Poppy) v1.0 with Meta Muse, Shopify and 100+ partners — yenkel · 2026-11-14
- Gensyn launches evidence-mining agent contest with $7,250 prize pool — benfielding · 2026-11-14
- 'Peak prompt engineering': full prompt for having an LLM draw a JS-only masterpiece — adonis_singh · 2026-11-14
- Superconductor adds Loops MCP connector so coding agents can manage email campaigns — sergeykarayev · 2026-11-14
- Dev records 2 hours of manual Neovim coding to get PRs accepted — yacineMTB · 2026-11-14
- Grok Bot tests auto-approving purchases with $50-$5,000 spend limits — XFreeze · 2026-11-14