Custom Agent Evals Show 35B Model Beating 120B Model

Engineer pauliusztin demonstrates how to build custom benchmarks for any agent framework, finding a 35B model achieved 95% success versus just 53% for a 120B model, showing custom evals matter more than raw scale.

2026-10-07 ~ 2026-10-07 · 2 related posts

• Agent Evals 101: Benchmarking and Regression-Testing Agent Harnesses, 35B vs 120B — blaizedsouza · 2026-10-07
• 35B Beats 120B on Custom Coding Agent Benchmark, 95% vs 53% Success Rate — blaizedsouza · 2026-10-07