Basalt Labs Accused of Outrageous Benchmark Fraud
WithoutReason1729 · reddit · 2026-07-18
A Reddit post is calling out the "outrageous scam" by Basalt Labs.
The poster claims the company stated they achieved 99.44% on HLE using tools, but the reality seems problematic:
- The model they released is based on Qwen2.5-7B-Instruct
- The model actually displayed/served on their website is DeepSeek
- It looks like exaggerated marketing or even a bait-and-switch model swap
The original post is highly sarcastic, primarily accusing the company of going way too far with their benchmark marketing.
More from Fun
- Dev builds city-sized zombie apocalypse sim in 100 days, ships on Steam — gdechichi · 2026-10-26
- 'Lamp Swarm': a desk of lamp-shaped robots shows what physical AI work looks like — dee_hw · 2026-10-26
- Dev-community banter: recommending a resource to every newcomer — itsOmSarraf_ · 2026-10-26
- Elon Musk's reply to a kid asking what's on Mars: "You get to decide" — yunta_tsai · 2026-10-26
- Prompt fills an empty room with everything your AI knows about you — jindong_wang92 · 2026-10-26
- The "claude is a lesbian" T-shirt is real and out in the wild — JasonBotterill · 2026-10-26