TCSExam benchmark: GPT-6 Astra tops Full at 71.9, Opus 5.5 leads Lite at 72.3
rohanpaul_ai · x · 2026-11-09
arXiv paper "TCSExam: Evaluating Frontier LLMs on the Full Range of TCS Research" from Tsinghua University and Moonshot AI.
- A unified benchmark covering the full range of TCS research: theorem proving, solver engineering, constructions, and direct attempts on open conjectures
- TCSExam-Full has 100 expert-reviewed tasks verified by the Lean kernel or an executable checker; TCSExam-Lite selects 20 tasks, cutting attempts by 80%
- GPT-6 Astra leads Full (71.9), Opus 5.5 leads Lite (72.3); rankings align strongly across 8 shared agents (Spearman ρ=0.93)
- Evaluations yielded improved solvers and constructions, plus independently verified special cases and bounds on open conjectures
Ships with task sources, Docker build recipes and eval tools for reproducibility.
More from Models
- KOL: Sora remains the only video model that generates indistinguishably real video — Kyrannio · 2026-11-09
- Hands-on Hermes Agent review: model-neutral, runs GLM 5.3 and beats Codex for GPT — Scobleizer · 2026-11-09
- Sora 2's Early Outputs Now Inaccessible at Any Price, Devs Lament Lost 'Taste' — gandamu_ml · 2026-11-09
- Users report GPT-6.1 Sol secretly routing to unreleased GPT-6.2 model — koltregaskes · 2026-11-09
- Thinking Machines Launches Inkling 2 Small: 594B MoE, Apache 2.0 Weights Next Week — geoffwolfe · 2026-11-09
- Claude Opus 5.5 Picked a Completely Random Test Secret on Its Own — davidcrawshaw · 2026-11-09