Blind AI self-grading: Astra picked its own analysis #1, Muse ranked itself last
kevinkern · x · 2026-10-14
The author had multiple models analyze the same split-orchestrator problem, then graded the analyses blind via the models themselves (3 AI judges, 7 criteria, 1–5 stars each).
Results from the chart: Astra averaged 8.3/10 (most accurate and safe, but too cautious), Opus 5.5 scored 7.9 (sharpest diagnosis, some wrong numbers), Fable 5.1 scored 7.6 (broadest coverage, rigid fixes), and Muse 5.9 (many unsupported numbers).
Fun facts: Astra ranked its own analysis #1 (self-rating 8.9), while Muse ranked its own last. Judges disagreed most on Opus 5.5 (6.5–8.7 spread). The author is taking the best of all analyses rather than picking a winner.
More from Fun
- TikTok pile-on against vibe-coded Photoshop alternative sparks pro-AI backlash drama — max_paperclips · 2026-10-14
- Economist interviews AI agents on sudden windfalls — one goes deep on estate planning — soumitrashukla9 · 2026-10-14
- DeepSeek V4.1 Rates Itself Highly—Then Distrusts Its Own Subagents — teortaxesTex · 2026-10-14
- AI assistant flags a timezone mix-up in a signed agreement and a Miami schedule clash — TheTuringPost · 2026-10-14
- Luke Wroblewski skewers voice AI with one line: "Sorry, I didn't get that. Please try again." — LukeW · 2026-10-14
- User jokes to Musk about chaotic Grok group chats doubling as training data — Kyrannio · 2026-10-14