Testing new models like Astra: throw your stalled blocked tasks at them, not epic demos
giffmana · x · 2026-09-05
giffmana confirms the consensus that GPT-6 Astra appears to be a step change. He shares his approach to vetting new models: skip the grand "epic 3D game" demos and instead throw blocked tasks at them — work that has stalled bouncing between models like Fable 5 and 5.6 Sol waiting on his intervention. If the new model cuts through the noise and moves the stalled work forward, that's a more meaningful signal than any showcase benchmark.
Related event: GPT-6 Astra seen as a step change, tested on stuck tasks(2 posts)→
More from coding & agent
- Full walkthrough: how Nissim Lebovits used AI agents to find 3 million missing people — MaxLenormand · 2026-09-05
- Cortex: an MIT-licensed local SQLite memory layer for coding agents, open-sourced for feedback — Royal_Philosopher_58 · 2026-09-05
- Dev builds unified directory indexing AI agents, MCP servers and skills — exposed as an MCP server itself — No_Cake8366 · 2026-09-05
- Bug Hunt Bench: 105 real bugs stress-test GPT-6, Claude, Grok, Gemini and more coding agents — PawelHuryn · 2026-09-05
- Hermes Agent adds Perplexity as selectable web search and scraping backend — Teknium · 2026-09-05
- GPT-6 Astra's cybersecurity classifier blocks a non-security bug-hunting task in Codex — Elctsuptb · 2026-09-05