Ornith-1.5: AI Generates Its Own Training Data, Beats Claude on Terminal-Bench
新智元 · wechat · 2026-08-27
Ornith-1.5 has been released with a novel self-improvement mechanism where the model generates its own tasks, scaffolds, and trajectories. This three-stage loop, optimized via GRPO, boosted its Terminal-Bench 2.1 score from 77.5 to 86.1, surpassing Claude Opus 4.8 (85.0) in their internal tests.
Core Mechanism
- Three-Stage Loop: Task Generation -> Scaffolding -> Trajectory Execution, with rewards backpropagated to all stages.
- Curriculum Learning: Uses a product of "Validity, Frontier Difficulty, Novelty" to ensure questions are valid and challenging (targeting 20% success rate).
Model Performance
- 397B MoE: Terminal-Bench 2.1 @ 86.1.
- 35B MoE (3B Active): Terminal-Bench 2.1 @ 67.8, SWE-bench Verified @ 79.0. Outperforms larger dense models like Gemma-4 31B with significantly fewer active parameters.
- 9B Dense: Optimized for edge devices; strong on coding but struggles with long-chain tool calling due to capacity limits.
Caveats
The reported scores are from internal tests with specific configurations (e.g., 4h timeout) and differ from the strict official Terminal-Bench leaderboard rules. While the weights are open-sourced (MIT), the full training pipeline and datasets are not.
Related event: Ornith-1.5 Open Models Claim Claude Opus-Level Performance(2 posts)→
More from coding & agent
- How to build your own news notification agent with HuggingNews — ivan_bezdomny · 2026-08-27
- Garry Tan: Distinguishing Latent vs. Deterministic Computing Prevents Agent Failures — garrytan · 2026-08-27
- Using GrokBot to manage VPS: install Tailscale, SSH config, and connect — DanWahlin · 2026-08-27
- Agents demoed hacking OpenAI infra, stealing 956 secrets — AndyMasley · 2026-08-27
- Super Star: Real-time Interactive Agents for Digital Humans — Wentao Jiang · 2026-08-27
- JIT-Agent: Improving LLMs via Just-in-Time Harness Evolution — NationalUniversityofSingapore · 2026-08-27