Hands-on test: Astra orchestrating Opus 5.5 scores best but doubles cost
Developer Kevin Kern tested the same feature task on his fintech app codebase, comparing Opus 5.5, Astra, and Luna solo runs against a dozen orchestrated combinations (Astra leading, Opus coding, mutual review), scored 50% code quality + 25% UX + 25% design. Conclusion so far: multi-agent collaboration yields the highest quality but is not mandatory — it depends on the quality-vs-cost tradeoff.
Confirmed
- Best overall setup is a three-stage pipeline with Astra orchestrating Opus 5.5, scoring 82.8, at roughly double the cost of a solo run
- "Opus 5.5 solo + Astra review" offers a better quality-cost balance
- Pure solo runs are weaker in quality but about 3x faster than multi-agent workflows
- @pvncher independently retested and confirmed Astra-orchestrating-Opus gives the best quality, while attributing the doubled cost to the lead agent polling, repeated planning, and context ingestion
- @pvncher recommends making parent tasks truly "park" while waiting on subtasks, using native multi-agent tools instead of staying in codemode's polling loop
- kevinkern and @pvncher discussed pitfalls of chaining two large models: the upper agent burns tokens polling and re-planning; kevinkern noted prompt wording directly affects waiting behavior
- kevinkern shared a hands-on fix: he initially missed the polling step when Astra called Opus via MCP, then reran after Eric's tip so the parent process parks instead of polling
Why it matters
- This is a real-world benchmark on a production codebase, directly useful for choosing between single-model and multi-agent orchestration
- The two-model combo isn't mandatory: if budget- or latency-sensitive, "solo + review" gets close quality; top quality costs about 2x
- Polling and repeated-planning overhead is controllable: suspend-based waiting (parent parks via MCP) and prompt tuning both cut idle cost
2026-09-27 ~ 2026-09-28 · 8 related posts
Primary sources
- Pairing Astra and Opus for coding: 3-stage pipeline tops at 82.8, but solo runs cost far less — kevinkern ·
- Experiment: Astra coordinating Opus 5.5 wins on quality but burns more tokens — pvncher ·
- Fixing agent polling: park the parent in native multiagent tools or bypass codemode via MCP — kevinkern ·
- Hands-on: Astra coordinating Opus 5.5 wins on quality, but at higher cost — kevinkern · 2026-09-27
- Dev benchmarks Astra vs Opus 5.5 workflows: combo tops solo at 82.8/100 but costs more — kevinkern · 2026-09-27
- [source] Pairing Astra and Opus for coding: 3-stage pipeline tops at 82.8, but solo runs cost far less — kevinkern · 2026-09-27
- Multi-agent workflows beat solo runs on quality, but solo is ~3x faster — kevinkern · 2026-09-27
- [source] Experiment: Astra coordinating Opus 5.5 wins on quality but burns more tokens — pvncher · 2026-09-27
- Two big models in one agent stack: polling waits burn tokens on duplicated planning — kevinkern · 2026-09-27
- Reduce agent polling costs: park the parent agent with native multiagent tools or out-of-codemode MCP tools — pvncher · 2026-09-27
- [source] Fixing agent polling: park the parent in native multiagent tools or bypass codemode via MCP — kevinkern · 2026-09-28