User's private benchmark finds Astra tiers degraded ~10x from release quality
PM__me_sth · reddit · 2026-09-19
A Reddit user shared a private benchmark they've run for two years, revealing what they see as major quality/quota degradation in Astra tiers: Extra High now delivers about 10% of its release-time value, while High, Medium and Light all dropped from 1 to 8-10 on their scale — roughly a 10x loss.
The benchmark is simple: 60 short sentences must be matched against 600 video files, with the AI extracting first/middle/end frames at 480p to justify matches and iterating as needed. It tests multimodal retrieval and judgment in practice.
The author bluntly calls out the provider for quietly degrading service and uses the private benchmark as a canary. Figures are one user's measurements, not independently verified.
More from Models
- Jev is (almost certainly) just an LLM returning a single token: an explainer — saurabhtwq · 2026-09-19
- A writer on Claude, likely the unreleased Opus 5 — Sauers_ · 2026-09-19
- LLMs Are Smart but Not Sharp: A New Qualitative Lens on Model Progress — jmugan · 2026-09-19
- Moonshot's Kimi K3 lands on Amazon Bedrock with 1M-token context and prompt caching — emmanuelvivier · 2026-09-19
- Gemini's 11-option model picker draws user ire: 'If it's so smart, figure it out' — Fine_Vermicelli2826 · 2026-09-19
- ChatGPT Searched 74 Websites to Answer a Daylight Savings Question — dioscuri · 2026-09-19