PostTrainBench agent trajectories downloaded 35k+ times in a month
maksym_andr · x · 2026-09-06
The PostTrainBench-Trajectories dataset from the AI Safety and Alignment Group at ELLIS Institute Tübingen and MPI-IS saw 35k+ downloads last month.
- The benchmark measures CLI agents' ability to autonomously post-train base LLMs: each agent gets a base model, an eval script, and 10 hours on one NVIDIA H100 80GB to improve benchmark performance via SFT, LoRA, RLHF, data-generation prompting, or any strategy it chooses.
- Base models: Qwen3-1.7B/4B-Base, SmolLM3-3B, Gemma-3-4B-PT. Benchmarks: AIME 2025, ArenaHardWriting, BFCL, GPQA, GSM8K, HumanEval, HealthBench.
- Each run ships sanitized full agent traces, metrics, and contamination judgments.
More from Research
- Slow Clinical Trials Break AI's Feedback Loop in Drug Development — clarejtbirch · 2026-09-06
- GPT-6 saturates RuneBench after just 6 months; author builds harder swarm tasks — SchoeneggerPhil · 2026-09-06
- Harness-of-Harness carries state across runs: 71.52 vs 58.24 on long-horizon coding — rohanpaul_ai · 2026-09-06
- DeepMind paper shows cheating spreading like an epidemic across ~100 AI agents — jackclarkSF · 2026-09-06
- Context Compaction Theory: first formal proof linking agent compaction to communication complexity — lateinteraction · 2026-09-06
- Last theorem on Freek Wiedijk's famous list has been formalized — satnam6502 · 2026-09-06