T3 Code blind audit: Gemini 4 Argon scores 3.95 with 51% false claims, Opus 5.5 tops at 8.40
theo · x · 2026-10-15
theo of T3 Code ran a blind, read-only architectural audit of the same commit across six models, scored by 4 judges on 8 weighted criteria with claims checked against the code. Claude Opus 5.5 won at 8.40 (5% false-claim rate), followed by Sonnet 5.5 (7.88), GPT-6.1 Sol (7.66), GPT-6 Astra (7.11), and Meta Muse Wonder Full (7.04). Gemini 4 Argon, even at its highest effort, scored just 3.95 with 47 of 92 checked claims marked false (51%). theo concludes it is "outright unusable for real code work," aggressively hallucinating how systems behave.
More from Models
- New models from Google and Meta crack the frontier AI duopoly — danshipper · 2026-10-15
- NVIDIA teases its own model launch amid a week packed with releases — TheOyinbooke · 2026-10-15
- Silence Is Billable on Voice Models: Even Dead Air Costs Money — sophiamyang · 2026-10-15
- NCM: new text generation paradigm up to 31x faster than diffusion LMs with better quality — simonguozirui · 2026-10-15
- Gemma 4 26B priced at $0.07/M input tokens, ~40% below comparable providers — testingcatalog · 2026-10-15
- Epoch AI: GPT-6 Astra tops frontier models in Slay the Spire 2 but still loses to top humans — sandersted · 2026-10-15