GPT-6.1 Sol third-party reviews: near-Astra performance at a fraction of the cost
After OpenAI released GPT-6.1 Sol, third-party reviews converged on a common verdict: performance close to the flagship GPT-6 Astra at a fraction of the cost. Artificial Analysis found Sol extends OpenAI's cost-efficiency frontier — $0.72 per Intelligence Index task at max effort — with per-task cost about 30% lower than GPT-6 Sol. Independent developers largely agree, though some benchmark results diverge.
Confirmed
- Pawel Huryn planted 105 bugs in 2 real code repositories (Bug Hunt Bench, blind, single-prompt): at max/xhigh, GPT-6.1 Sol fixed 44 bugs for $6.56 vs GPT-6 Astra's 45 for $33; all effort levels published (n=3 for max/xhigh, n=2 otherwise, more reruns ongoing); he sees no reason to use Astra given the cost
- adonissingh's eyebench-v3: Sol ranked second, near Astra at 3.8x cheaper, 1/8 the price of Opus-5.5; Pareto-optimal on cost, but output token efficiency below Astra at every effort level
- cedricchee found 3D generation near Astra quality with 30% faster speed, +4 points on AA Index over GPT-6 Sol; codestantine reported Sol beating Astra on a self-built low-resource translation benchmark (relayed by nickbaumann)
- danielmckinn0n on GamowLabs RareBench: Sol nearly matches Astra at 1/4 the price, on the cost Pareto frontier
- Artificial Analysis published evaluations for all effort levels (Low to Max) and updated Intelligence Index to v4.3 (AutomationBench-AA replaces old items)
- Reddit user therealjerseytom says it "lives up to the hype": handles challenging work like 6 Astra at much lower token cost, especially when rework is needed
Unconfirmed
- BridgeBench shows Sol only 3 points above GPT-6 Sol and 82 behind GPT-6 Astra, sharply diverging from OpenAI's "near Astra-level intelligence" claim and other reviews; awaits independent verification
- Wsz2020 cautions the "88% cheaper than Opus 5.5 max" headline is misleading given Opus's different pricing structure
- A joke Runescape Bench (relayed by banteg) ranks Sol second at 10% of Astra's price — limited reference value
- User chogku claims a GPT-6.1 Sol + Opus 5.5 combo approaches Astra at far lower cost (relayed by rudrank); personal opinion
Why it matters
Pawel Huryn stresses that unlike GPT-6 Sol, seen as a nerfed GPT-5.6 Terra, GPT-6.1 Sol is a genuine capability gain. If the low-cost high-performance result holds across more scenarios, it could reshape the price-performance landscape of frontier models and squeeze flagships like Astra out of everyday coding and reasoning workloads.
2026-09-30 ~ 2026-10-02 · 21 related posts
- Episode 1: OpenAI Launches GPT-6 Sol and Luna at Half Price; Big Hallucination Gains but Not a Clean Sweep(2026-09-23, 92 posts)
- Episode 2: GPT-6 Sol/Luna launch splits opinion: half-price but dumber?(2026-09-23, 9 posts)
- Episode 3: LMArena Opens GPT-6 Sol and Luna for Public Testing(2026-09-23, 2 posts)
- Episode 4: Four New Models Reshape the Cost-Performance Frontier, Opus 5.5 Tops Intelligence Index(2026-09-23, 2 posts)
- Episode 5: OpenAI and Anthropic Slash Prices Within 90 Minutes, Kicking Off a New Model Price War(2026-09-23, 7 posts)
- Episode 6: Tests Show GPT-6 Sol Scales Nearly Linearly With Effort(2026-09-23, 3 posts)
- Episode 7: GPT-6 Astra Tops Bug Hunt Bench While Sol Version Regresses(2026-09-24, 2 posts)
- Episode 8: Leaks Map GPT-6 Codenames to GPT-5.6 Versions(2026-09-24, 3 posts)
- Episode 9: Bug Hunt Bench Ranks Frontier Coding Models on 105 Real Bugs with 200x Cost Gap(2026-09-24, 3 posts)
- Episode 10: GPT-6 Sol and Luna Launch With Aggressive API Pricing(2026-09-25, 2 posts)
- Episode 11: OpenAI's efficient GPT-6 models seen as freeing compute for bigger frontier models(2026-09-25, 2 posts)
- Episode 12: GPT-6 Sol Underestimated: 1.6x Robot Task Gains at 47% Lower Cost(2026-09-25, 2 posts)
- Episode 13: GPT-6.1 Sol Launches with Astra-Level Intelligence at a Fifth of the Price(2026-09-29, 32 posts)
- Episode 14: GPT-6 Astra Ultrafast Reportedly 8x Faster in Codex(2026-09-30, 2 posts)
- Episode 15: GPT-6.1 Sol third-party reviews: near-Astra performance at a fraction of the cost(2026-09-30, 21 posts)
Primary sources
- GPT-6.1 Sol costs a quarter of GPT-6 Astra per task, pushing OpenAI's cost-efficiency frontier — ArtificialAnlys ·
- GPT-6.1 Sol fixes 44 of 105 planted bugs for $6.56, matching Astra at a fraction of the cost — PawelHuryn ·
- Artificial Analysis posts full GPT-6.1 Sol evals, rolls out Intelligence Index v4.3 — ArtificialAnlys ·
- GPT-6.1 Sol near-Astra quality in 3D work while running ~30% faster — cedric_chee · 2026-09-30
- GPT-6.1-Sol Takes #2 on eyebench-v3, ~8x Cheaper Than Opus-5.5 — adonis_singh · 2026-09-30
- Cost Pareto-dominant, but second only to Astra on output-token efficiency — adonis_singh · 2026-09-30
- GPT 6.1 Sol Only +3 on BridgeBench, 82 Points Behind Astra: Benchmaxing Suspected — RexDouglass · 2026-09-30
- GPT-6.1 Sol scores 44 on real repos for just $6.56, near-top accuracy at 1/5 the cost — PawelHuryn · 2026-09-30
- [source] GPT-6.1 Sol fixes 44 of 105 planted bugs for $6.56, matching Astra at a fraction of the cost — PawelHuryn · 2026-09-30
- GPT-6.1 Sol lands #2 on 'Runescape Bench' at ~10% the price of Astra — banteg · 2026-09-30
- Bug Hunt Bench updates all effort tiers; GPT-6.1 Sol xhigh nearly free on 105 planted bugs — PawelHuryn · 2026-09-30
- Early GPT-6.1 Sol user report: similar work done at dramatically lower token cost than 6 Astra — therealjerseytom · 2026-09-30
- GPT-6.1 Sol + Opus 5.5 combo called near-Astra quality at a fraction of the cost — rudrank · 2026-10-01
- GPT-6.1 Sol beats Astra on low-resource translation, dev benchmark shows — nickbaumann_ · 2026-10-01
- GPT-6.1 Sol benchmarks across all effort levels land, making GPT-6 Astra hard to justify — PawelHuryn · 2026-10-01
- GPT-6.1 Sol Benchmarked Across All Effort Levels; Atra Faster but Pricier — PawelHuryn · 2026-10-01
- GPT-6.1 Sol cuts cost per task ~30% further, now roughly a third of GPT-5.6 Sol — ArtificialAnlys · 2026-10-01
- [source] GPT-6.1 Sol costs a quarter of GPT-6 Astra per task, pushing OpenAI's cost-efficiency frontier — ArtificialAnlys · 2026-10-01
- [source] Artificial Analysis posts full GPT-6.1 Sol evals, rolls out Intelligence Index v4.3 — ArtificialAnlys · 2026-10-01
- Opus 5.5 high vs GPT-6.1 Sol max: AA data shows Sol's value edge is mostly latency, not intelligence — Wsz2020 · 2026-10-02
- GPT-6.1 Sol nearly matches Astra at a quarter of the price on RareBench — danielmckinn0n · 2026-10-02
3 near-duplicate retellings: cedric_chee · adonis_singh · PawelHuryn