TB-fn Benchmark Retests Top Terminal Models, Exposing Suspected Leaderboard Gaming

Fidian built TB-fn, a variant of Terminal-Bench, retesting roughly the top 20 models from the Terminal-Bench 2.1 leaderboard. The score gap widened from about 15 points on TB-2.1, with only OpenAI and Anthropic models remaining in the frontier tier and several previously high-scoring models suspected of leaderboard gaming.

Confirmed

• TB-fn was built by Fidian on top of Terminal-Bench, retesting about the top 20 models from the TB-2.1 leaderboard.
• TB-fn redesigns the original 89 tasks: adding steps, tightening requirements or requiring more complex algorithms, removing shortcut solutions, and ensuring validators only check intended criteria.
• Models that scored similarly on TB-2.1 diverged sharply on TB-fn, with average task difficulty rising.
• Per reviewers, GPT-5.6 Sol and Opus 5 stayed ahead; Grok 4.5, Kimi K3 and GLM-5.2 dropped 6-11 points.

Why it matters

• TB-fn directly targets shortcut solutions and solution memorization on leaderboards, offering a more credible measure of real terminal/agent model capability.
• The large gaps for some vendors' models between old and new benchmarks highlight industry concerns about benchmark gaming, potentially affecting leaderboard credibility and model selection.

Note: most posts in this cluster are reposts; primary data should come from the Terminal-Bench team's original release.

2026-08-26 ~ 2026-08-28 · 6 related posts

• New TB-fn Benchmark Exposes Model Performance Gaps on Terminal Tasks — abeirami · 2026-08-26
• New TB-fn Benchmark Reveals Significant Drops in Terminal Model Rankings — abeirami · 2026-08-27
• TerminalBench Deep Dive: GPT-5.6 Leads, Many Models Drop in Rank — abeirami · 2026-08-27
• TB-fn benchmark reveals score drops for several models, exposing potential benchmark overfitting — burny_tech · 2026-08-28
• [source] Terminal benchmark TB-fn: OpenAI and Anthropic maintain clear lead — abeirami · 2026-08-28

1 near-duplicate retellings: abeirami