SOOFI Accused of Inventing Metrics to Inflate Capabilities
JJitsev · x · 2026-07-19
The author dismisses SOOFI's "frontier-level / champion" claims as baseless. The project allegedly invented a "capability index" that no one else uses to score itself, subsequently claiming to be on par with or even superior to the original Nemotron-3-Nano.
He also points out the contradiction with the report's "transparency" narrative. Drawing conclusions based on custom metrics and larger training volumes—without transparently stating these premises—is highly misleading to readers.
Related event: European Open-Source Model SOOFI Accused of Eval Leakage and Overhype(11 posts)→
More from Models
- GPT-6.1 Terra Now Testable on LMArena in Battle and Agent Modes — arena · 2026-10-27
- OpenAI Launches GPT-6.1 Terra at $1/M Input Tokens, ~9x Cheaper per Task Than Opus 5.5 — VraserX · 2026-10-27
- GPT-6.1 Terra Benchmarks: Sweeps All Six Tests, SWE-bench at 76.2% — mark_k · 2026-10-27
- GPT-6.1 Terra lands: early users praise speed, value and subagent performance — BenBajarin · 2026-10-27
- Leaked GPT-6.1 Terra benchmarks: cheap but 'boring', still far behind Astra and Claude — Angaisb_ · 2026-10-27
- Artificial Analysis Ships Intelligence Index v4.3.2 With GPT-6.1 Terra Comparison — ArtificialAnlys · 2026-10-27