Running a prompt 30 times until your favorite model wins doesn't prove the other one sucks
VraserX · x · 2026-11-06
VraserX calls out a common bias in model comparisons: people rerun the same prompt 30 times until their favorite model succeeds, then post the other model's worst attempt as proof it sucks. He urges the community to stop this practice.
More from Fun
- Canva's 'We miss you' push notification lands next to 44 unread emails — alexmacgregor__ · 2026-11-06
- Only Opus 4.8 got paranoid and refused a hypothetical about its crooked smile — repligate · 2026-11-06
- The most obvious AI slop ever, yet Instagram comments call it beautiful — flowersslop · 2026-11-06
- A single desktop B70 runs an entire AI livestream radio, generating music in real time — prince_pringle · 2026-11-06
- Sarcasm: programming is dead, as long as you still have great engineers — RexDouglass · 2026-11-06
- Developer's follower count spikes 827 in one day, suspects a bot follow campaign — haydendevs · 2026-11-06