Local Gemma 4B Model Fails High-Difficulty Agent Benchmark

draginol tested the gemma-4-E4B-it-qat-q40 model on a local agent benchmark, where it scored only 2 out of 5 and failed the highest-difficulty Extended multi-step agent tasks, suggesting small local models still fall short on complex agentic work.

2026-11-08 ~ 2026-11-08 · 2 related posts

• Dev benchmarks Gemma 4B on local agent tasks: scores 2/5, fails Extended mode threshold — draginol · 2026-11-08
• Local 4B model fails Extended agent benchmark: 2/5 tasks at 61.5 tok/s — draginol · 2026-11-08