Gemini CLI silently swapped 3.7 Flash for 3.5; log your actual API calls
PawelHuryn · x · 2026-09-04
The author recounts a benchmark debugging saga: before Antigravity CLI launched, he ran 3.7 Flash through the Gemini CLI, but every API call silently went to gemini-3.5-flash despite the startup line saying 3.7. The cause: the CLI had a built-in model list, and unknown flash models were quietly replaced with the default.
He only caught it by logging every API call, then put a small gateway between the CLI and Google to rewrite the model name — 498/498 calls verified after that. Lesson: the requested model is not the setting; log what actually got called.
Related event: Bug Hunt Bench: 105 Hidden Bugs, No Model Finds Them All, Cost Gap Nears 8x(6 posts)→
More from coding & agent
- GPT-6 Astra tip: delete your AGENTS.md and start fresh, old cruft will haunt you — i_dg23 · 2026-09-04
- Cognition brings GPT-6 Astra to Devin: near-Fable 5 performance at 64% lower cost — sandersted · 2026-09-04
- Vibe-Coded a CRM Overnight with Cursor to Run Nigeria's Private School Owners Conference — saheedniyi_02 · 2026-09-04
- CMU Prof: Code Review Won't Survive AI-Generated Code — and That's a Chance to Redesign It — CharlotteHase · 2026-09-04
- From 2% success to mainstream: Mind2Web author on how far computer-use agents have come — hhsun1 · 2026-09-04
- Matt Shumer's GPT-6 Astra review: Manager Loop sustains a week-long Manhattan build in Unreal — mattshumer_ · 2026-09-04