Testing gemini-4-neon with a DIY PowerShell agent harness: "benchmarks are for tourists"
dfinke · x · 2026-11-19
Developer dfinke says Google just dropped gemini-4-neon (unverified) and argues the best way to test a new model is pointing your own tools at it — "Benchmarks are for tourists." His PSAISuite PowerShell module offers a one-cmdlet harness: New-AgentHarness takes a model, tools, and a prompt. A screenshot shows gemini-4-neon correctly enumerating and summarizing 8 PowerShell tool scripts (system info, text processing) and even noticing fib.ps1 wasn't registered in tools.json. Install: Install-Module PSAISuite.
More from coding & agent
- Garry Tan's "Personal AGI" is a folder of markdown files — 220,000 pages, 400x his 2013 output — jasonkneen · 2026-11-19
- Same weights, 3x score: Gemini 4 Neon jumps 12.8%→39% on SWE-sweep by swapping the harness — Delicious_Character6 · 2026-11-19
- ESP-Skainet: Open-Source Framework for Fully Offline Voice AI on ESP32 — alexcovo_eth · 2026-11-19
- Ploten ships an MCP server that live-edits open Power BI Desktop reports — plotenhq · 2026-11-19
- Merge's Agent Handler adds credential passthrough: run tools without storing user tokens — shensi · 2026-11-19
- Gemini 4 Neon called best everyday coding model, beating Opus 5.5 — inductionheads · 2026-11-19