Reviewer recomputed 7,000 benchmark answers — then found a 433-pound dog
ziv_ravid · x · 2026-11-10
A reviewer recomputed all 7,000 reference answers of a benchmark on reading numbers across cultures, and every one matched. But when he actually read the questions, he found absurd examples — like a question claiming each sausage cost $38,320, and a 433-pound dog. Author zivravid says the full post with screenshots and links lives on his information bottleneck website, with more to come next year.
Related event: Reviewer Verifies 7,000 Benchmark Answers, Finds Absurd Questions(2 posts)→
More from Fun
- Polymarket opens market on Jensen Huang's potential $81B divorce settlement — gethackteam · 2026-11-10
- Claude 3 Sonnet co-designs abyssal sea turtle that sings with its breath — repligate · 2026-11-10
- Claude Fable 5.2 builds first cherry-wood Minecraft house in blogger's test, and it looks great — Angaisb_ · 2026-11-10
- Sentry CEO zeeg reveals LLM-built game with voxel building and fully destructible terrain — zeeg · 2026-11-10
- A 1996 LA Times column predicted the Internet would fail—and replace 'web' with 'agents' today — omooretweets · 2026-11-10
- 'The Day He Woke Up a Loser': X users riff on a hypothetical Jensen Huang movie — JacquesThibs · 2026-11-10