Dev: I trust Discord vibe checks over benchmarks now — and that's terrible science
burny_tech · x · 2026-11-18
burnytech says his trust in "benchmaxxed" leaderboards has collapsed and he now relies on collective X/Discord vibe checks — then self-mockingly notes that this is data with zero statistical rigor, plenty of bias and selection effects. A candid take on how community sentiment is replacing public benchmarks.
More from Models
- PenaltyBench: Gemini 4 Carbon beats VAR referees at calling penalties, 63.3 vs 62.3 — ziqiao_ma · 2026-11-18
- Tencent quietly lists Hy4-flash model page on Hugging Face — BreakfastFriendly728 · 2026-11-18
- Claude models 'discover' body parts only when touched, says researcher — repligate · 2026-11-18
- GPT-6.2 Sol hands-on: Fast mode finishes tasks noticeably quicker, says KOL — op7418 · 2026-11-18
- User posts receipts: GPT 6.2 Astra catches what Opus 5.5 missed — soumitrashukla9 · 2026-11-18
- "Only 3 frontier AI companies now — and it's not winner-take-all" — repligate · 2026-11-18