Gemini 3.8 Flash eval: strong at numerical methods, mogged by Qwen 3.8 27B on checkpoints
AnuranBuilds · x · 2026-09-03
PhyseraAI benchmarked Gemini-3.8-Flash on its TB bench across 5 random tasks: it excels at deriving and implementing coherent numerical methods, but is inconsistent on interacting edge-case semantics, overbuilds static-analysis solutions while missing the hardest coverage cases, and long trajectories often mean speculative scope expansion rather than better outcomes. Another developer confirms Gemini's random-quality checkpoints make it lose badly to smaller models like Qwen 3.8 27B on checkpoint/sequence-based work.
More from Models
- MOSS SFT vs Flux 3 Klein vs SD 3.0: Same-Prompt Comparison Exposes SD3's Age — ZootAllures9111 · 2026-10-18
- Quick take: GPT-6 matches Opus 5.5 at coding, far ahead at computer use — _vztu · 2026-10-18
- Open models now above Sol-level, outpacing expectations as community awaits V4.1 Pro — yacineMTB · 2026-10-18
- Dev's model vibes: Opus 5.5 writes best, Sol 6.1 best value, Sonnet skippable — ricklamers · 2026-10-18
- Codex took 3 days, Opus 5.5 nailed it in an afternoon — banned user's video protest — dotey · 2026-10-18
- Yacine MTB: Users will remember how Anthropic treated them once intelligence proliferates — yacineMTB · 2026-10-18