Questioning Exaggerated Marketing in Open-Source Models
JJitsev · x · 2026-07-19
The author calls out the marketing tactics of certain open-source projects, arguing that using cherry-picked evaluation results to claim parity with top-tier open-source models is highly misleading to both the public and policy discussions.
An attached image showing an accuracy comparison across multilingual tasks—including models like Teuken-7B and Mistral 7B—serves to highlight the disconnect between marketing claims and actual performance.
Related event: European Open-Source Model SOOFI Accused of Eval Leakage and Overhype(11 posts)→
More from Models
- Artificial Analysis breaks down all sub-evaluations in Intelligence Index v4.3.2 — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra gains ~20 points on Terminal-Bench 4.0, rivalling GPT-6.1 Sol — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra cuts hallucination rate by 50 points while lifting accuracy 2 points — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Leaked Terminal-Bench 4.0 chart shows GPT-6.1 Terra leading at lowest cost, unverified — Angaisb_ · 2026-10-27
- OpenAI brings GPT-6.1 Terra to the API and Codex for fast, cheap computer use — OpenAIDevs · 2026-10-27