Ornith-1.5: AI Generates Its Own Training Data, Beats Claude on Terminal-Bench
新智元 · wechat · 2026-08-27
Ornith-1.5 has been released with a novel self-improvement mechanism where the model generates its own tasks, scaffolds, and trajectories. This three-stage loop, optimized via GRPO, boosted its Terminal-Bench 2.1 score from 77.5 to 86.1, surpassing Claude Opus 4.8 (85.0) in their internal tests.
Core Mechanism
- Three-Stage Loop: Task Generation -> Scaffolding -> Trajectory Execution, with rewards backpropagated to all stages.
- Curriculum Learning: Uses a product of "Validity, Frontier Difficulty, Novelty" to ensure questions are valid and challenging (targeting 20% success rate).
Model Performance
- 397B MoE: Terminal-Bench 2.1 @ 86.1.
- 35B MoE (3B Active): Terminal-Bench 2.1 @ 67.8, SWE-bench Verified @ 79.0. Outperforms larger dense models like Gemma-4 31B with significantly fewer active parameters.
- 9B Dense: Optimized for edge devices; strong on coding but struggles with long-chain tool calling due to capacity limits.
Caveats
The reported scores are from internal tests with specific configurations (e.g., 4h timeout) and differ from the strict official Terminal-Bench leaderboard rules. While the weights are open-sourced (MIT), the full training pipeline and datasets are not.
Related event: Ornith-1.5 Open Models Claim Claude Opus-Level Performance(2 posts)→
More from coding & agent
- Vercel Goes Full Backend: Python GA, Containers, and Daemons — cramforce · 2026-10-16
- WorldMemArena Accepted at NeurIPS: Memory Storage ≠ Memory Use — xwang_lk · 2026-10-16
- Vercel Ship Demos Container Deploys for Go, Ruby, Java and More — edwin · 2026-10-16
- TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s — QGallouedec · 2026-10-16
- Databricks engineers land impactful PRs on day 2 with AI dev tools, vs week 3 before — Yuchenj_UW · 2026-10-16
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16