User challenges Arena benchmarks: simple SVG prompt turned into full interactive website
Angaisb_ · x · 2026-09-17
Angaisb questions Arena benchmark fairness: a plain 'make an SVG of a PS5 controller' prompt yielded a full interactive website with internals, suggesting Arena's system prompts over-prompt models.
More from Models
- Noam Brown: models may perform their chain of thought; alignment must be solved — infoxiao · 2026-09-18
- Jev fails as an LLM scorer on OntBench: rates almost everything positively, contradicting human and Codex ratings — amplifiedamp · 2026-09-18
- Independent eval puts new model Jev at Terra no-think level, roughly on par with Luna-xhigh — tokenbender · 2026-09-18
- Hugging Face adds zero-shot text classification pipeline to its API, no training needed — joeddav · 2026-09-18
- Base Labs launches open-weight AI safety partnership with Hugging Face and Goodfire — TechCrunch AI · 2026-09-18
- Karpathy lays out the race for a small on-device LLM "cognitive core" as JEV positions itself as a first step — Paimaamu · 2026-09-18