TB-fn benchmark reveals score drops for several models, exposing potential benchmark overfitting
burny_tech · x · 2026-08-28
Fidian released TB-fn, a variant of Terminal-Bench, revealing significant score drops for several models and exposing potential overfitting. Grok 4.5, Kimi K3, and GLM-5.2 dropped by 6-11 points, while only OpenAI and Anthropic models remained stable at the frontier. GLM-5.3 Flash leads open weights but its gap to Sol max widened from 1.0 to 8.6 points. The benchmark also showed a 41% average increase in per-attempt cost across all models.
More from Models
- Image inputs now ~10,000 per dollar, but re-referencing an image doubles token cost — smtabatabaie · 2026-10-15
- "Argon doesn't trust me": researcher laments overzealous model refusal — thegautamkamath · 2026-10-15
- User finds Grok shockingly bad at finding X posts, prefers Opus with browser — JasonBotterill · 2026-10-15
- "Claude-Next" Reportedly Returns to Claude CLI and Web via Stealth Opus Routing — koltregaskes · 2026-10-15
- 'Thinking leakage': no-think RL gains in hybrid models just reuse think-mode skills — kaust-generative-ai · 2026-10-15
- Cline: Gemini 4 Argon saw 3x Claude's day-one adoption and 30% fewer tokens — JMateosGarcia · 2026-10-15