Subscription value test: Gemini ~26x vs OpenAI ~10.5x API credits; real-bug eval underway
PawelHuryn · x · 2026-11-19
Pawel Huryn shares subscription-value math: measured on a $99.99/mo plan, OpenAI's subscription yields 10.5x its cost in API credits, while early Gemini results suggest 25.2–26.2x. He is also running a frontier-model bug-fixing eval using real, previously unfixed bugs from early 2026 — only fully fixed bugs score, bug-maxing reports are excluded, and all models are judged blind by the same judge. Full results coming soon.
More from Models
- Anthropic allegedly profiles users and silently downgrades them to weaker models at full price — JafarNajafov · 2026-11-19
- Antigravity CLI 1.5.12 screenshot shows Gemini 4 Neon (Medium) model option — patloeber · 2026-11-19
- Daily digest: Xiaomi MiMo spinoff rumor, Tencent cuts Yuanbao spend, Gemini 4 Neon ships — 快鲤鱼 · 2026-11-19
- Branching a ChatGPT workspace wipes the agent's context, user complains — LucaAmb · 2026-11-19
- 0.5B ASR model hits 6.96% multilingual WER, beating Meta's 15x-larger omniASR — thisguyknowsai · 2026-11-19
- tinygrad publishes 10-411 week-1 assignment, testing LLMs one-shot on held-out test set — CatAstro_Piyush · 2026-11-19