LiveNerf hits ~1,000 stars tracking daily whether Opus 5.5 gets 'nerfed' post-release
TheOnlyVibemaster · reddit · 2026-10-02
A Reddit developer built LiveNerf, an open-source project that evaluates Opus 5.5 daily against a fixed benchmark to test whether models measurably degrade after release, instead of relying on anecdotes.
- The repo is approaching 1,000 stars, hit the Hacker News front page, and has drawn heavy scrutiny of its methodology.
- The project is still establishing a baseline; once enough data accumulates, statistical comparisons of later performance become possible. A finding of no degradation would also count as a result.
- Community criticism covers benchmark contamination, providers possibly detecting benchmark traffic, benchmark selection, and statistical methodology; the author welcomes issues.
- The author argues that since AI companies don't disclose model changes over time, the community should build its own measurement tools. Results will be posted once evidence emerges.
More from Models
- UI screenshot appears to show unreleased 'GPT-6.2 Sol' with reasoning-effort slider — paw_lean · 2026-11-19
- Kaggle lays out roadmap to fix AI benchmarks: relevance, saturation, verification, infrastructure — MeganRisdal · 2026-11-19
- Mistral Large 4 coding test: 2 hours of agentic work burned just €0.82 of quota — qtnx_ · 2026-11-19
- User runs GPT-6.2-Astra-Max on fast mode for 24h, burns only 20% of weekly limit — prd_008 · 2026-11-19
- Kling Image 4.0 debuts at #7 on Artificial Analysis, only model with native 8K output — anthara_ai · 2026-11-19
- Contra Labs launches Frontier Design Benchmark; stealth model Gestalt beats human gold-standard designs — yuwen_lu_ · 2026-11-19