pass@1 dead even, but Fable 5.1 wins pass@k over GPT 6 Astra
zainhas · x · 2026-09-05
Follow-up data from the same Terminal Bench 4.0 runs: GPT 6 Astra and Fable 5.1 are dead even on pass@1, but GPT 6 Astra is less impressive at pass@k for k>1, suggesting less headroom from multi-attempt sampling.
More from Models
- No usage reset on GPT-6 Astra launch day, developer calls out OpenAI — tobowers · 2026-09-05
- Debate: Models Fuzzily Recall Concepts, Not Text — SAE Features vs Edit-Distance Memorization — voooooogel · 2026-09-05
- Blogger feeds GPT6 Astra a PPT template, gets 30 conference slides with the right avatar — vista8 · 2026-09-05
- Dozens of GPT-6 Astra prompts collected via GPT 6 pro web search — vista8 · 2026-09-05
- Tencent Hunyuan Hy4 preview: 770B total/49B active, 1M context, Apache 2.0, day-0 vLLM — aftahi_ai · 2026-09-05
- Astra Draws Stunning Illustrations Stroke by Stroke, Not Generated — Tolopono · 2026-09-05