SOOFI Accused of Inventing Metrics to Inflate Capabilities
JJitsev · x · 2026-07-19
The author dismisses SOOFI's "frontier-level / champion" claims as baseless. The project allegedly invented a "capability index" that no one else uses to score itself, subsequently claiming to be on par with or even superior to the original Nemotron-3-Nano.
He also points out the contradiction with the report's "transparency" narrative. Drawing conclusions based on custom metrics and larger training volumes—without transparently stating these premises—is highly misleading to readers.
Related event: European Open-Source Model SOOFI Accused of Eval Leakage and Overhype(11 posts)→
More from Models
- Harness bug masked Gemini 4 Argon: rerun with Antigravity beats Astra, nears Opus 5.5 — vitaliychiley · 2026-10-27
- GPT 6.1 Terra vs Claude Sonnet 5.5: speed and value vs raw capability — teortaxesTex · 2026-10-27
- Harmonic Teases: AI Taught to Prove Math, 'Something Harder' Coming Soon — satnam6502 · 2026-10-27
- OpenAI quietly swaps Plus/Pro default to cheaper GPT-6.1 Terra and cuts Codex allowance — kimmonismus · 2026-10-27
- 'Never been this uninterested after an OpenAI model release' — theteknosaur · 2026-10-27
- Dev questions reports of Chinese models writing insecure code, citing thin methods and conflicts — dbreunig · 2026-10-27