Rerunning Gemini 4 Argon with Antigravity harness lifts FrontierCS 2 score ~15%
vitaliychiley · x · 2026-10-27
Auditing FrontierCS 2 traces, @wenhaocha1 found the Pi Harness ends runs whenever it hits a fallback (e.g. on security tasks) in 20% of runs, badly undercounting the model. Rerunning Gemini 4 Argon with the Antigravity harness raised its score from 40 to 54 (15% higher), beating Astra and sitting just behind Opus 5.5. The retweeter adds that Argon with agy performs differently than with any other harness — a reminder that benchmark scores can reflect harness bugs, not model capability.
Related event: Harness Flaw Undercounted Gemini 4 Argon; Rerun Nears Opus 5.5(6 posts)→
More from Models
- Bug Hunt Bench leaderboard: Sonnet 5.5 max tops at 51.3 by refusing to stop — PawelHuryn · 2026-10-27
- GPT-6.1 Terra scores 40.7 on Bug Hunt Bench at max, but collapses below that tier — PawelHuryn · 2026-10-27
- Fable 5.5 rumored to drop tomorrow, per developer teaser — daniel_mac8 · 2026-10-27
- A Qwen3.5-4B fine-tune is getting better at generating 3D scenes — ConfidentDinner6648 · 2026-10-27
- Enterprise benchmark: GPT 6.1 Sol High scores 94.8% at $1.07 per task, DeepSeek 1017 78.1% at $0.42 — emilahlback · 2026-10-27
- Cohere releases Qwen investigation traces for independent security verification — Cohere · 2026-10-27