Gemini 3.7 Flash Performance Drops Below Gemma Due to OpenRouter Backend Variances
mariofilhoml · x · 2026-08-14
The author found that the newer Gemini 3.7 Flash performed worse than Gemma on a structured text extraction task. Upon investigation, they discovered that when using Gemini models via OpenRouter for structured output, the underlying backend (Vertex vs. AI Studio) causes wildly different performance.
This serves as a warning for developers to account for backend discrepancies when running model evaluations.
Related event: Gemini 3.7 Flash Structured Output Varies by Backend(2 posts)→
More from Models
- Vercel Offers GLM 5.2 Model Free for eve Agents Until August 27 — cramforce · 2026-08-14
- Deepgram Crosses $100M ARR and Launches Flux TTS Voice Model — deepgramscott · 2026-08-14
- SemiAnalysis: DeepMind Overhaul Signals Gemini's Downfall, GCP Emerges as Winner — ben_j_todd · 2026-08-14
- Musk Offers More Free Usage and Resets Limits for Grok 4.6 Launch — EricBuess · 2026-08-14
- Grok 4.6 Tops GPQA Diamond Leaderboard with 94.9% Score — elonmusk · 2026-08-14
- a16z's Martin Casado Tests Grok 4.6: Impressed by Complex Coding and Long Tasks — elonmusk · 2026-08-14