HotDogBench tests 10 LLMs on 1,000 hot dog photos — all within each other's error bars
josh_wills · x · 2026-10-16
MotherDuck launched HotDogBench, a tongue-in-cheek benchmark: 10 models were shown 1,000 photos (239 hot dogs, 761 not) and asked the only question that matters — is it a hot dog?
- Too close to call: all models scored 969–985 out of 1,000, inside each other's error bars
- Winner Jev 1.14: 16 mistakes, 98.4% accuracy, fastest at 250 photos/s (5.1× the next model), 9.9¢ per 1,000
- Cheapest Perplexity Decider: just 1.7¢ per 1,000, but 20 false positives (called herring rolls and cheesesteaks hot dogs)
- Priciest GPT-6 Astra: $10.27 per 1,000 — 600× Perplexity for nine fewer mistakes
- Per-model failure cases are browsable, e.g. mistaking a painted hot dog cart or an alligator po' boy for a hot dog
More from Fun
- Viral wish: tap two iPhones to download anyone's Community Notes — yungcontent · 2026-10-17
- The internet is now clips from podcasts that never existed: AI slop designed to be noticed, not watched — yungcontent · 2026-10-17
- Aronchick: you can love ambitious demos and still demand they be real — aronchick · 2026-10-17
- AI Gets You 90% Done in 10 Minutes; the Last 10% Costs 4 Hours of Debugging — tech__unicorn · 2026-10-17
- AI slashed coding time — and it all went to founder dinners — signulll · 2026-10-16
- Tesla's Roadster 0-60 mph demo video is exactly one second long — yungcontent · 2026-10-16