PostTrainBench Author: 10-Hour GPU Cap Explains Benchmark Hill-Climbing
maksym_andr · x · 2026-11-20
Responding to criticism that post-training tasks share PostTrainBench's flaw—models merely hill-climb a single benchmark and end up distilling from other models—author maksymandr says the root cause is compute budget. As an academic lab, PostTrainBench capped runs at 10 GPU hours; any frontier lab or well-resourced org (e.g. Epoch AI) could scale it to thousands of GPU hours with multiple benchmark targets and see what happens.
More from Research
- RUC & Fudan release CoworkOS: agent runtime for long-horizon work with human edits, boosting benchmarks from 33.19 to 42.95 — jiqizhixin · 2026-11-20
- Team claims Codex cracked 35 open Erdős problems in 2 months, some Lean-verified — IgorCarron · 2026-11-20
- ForgeSystem paper proposes AI agents forging demand-specific software — Div_pradeep · 2026-11-20
- Accord: open-source dense metric mapping from a single camera and IMU — BE2R · 2026-11-20
- AltRA-Test: ~100 samples suffice to decide if LLM judges can replace humans — Technion · 2026-11-20
- TwinD hits 95.6% AUROC on face synthesis deepfakes, beating prior detectors — Andrii Yermakov · 2026-11-20