Running non-vision models is a pain now that evals mix in stray images
xeophon · x · 2026-09-28
xeophon complains that modern eval suites increasingly mix in one or two images per task, making it painful to run models without vision support. Tooling has to handle these mixed-modality tasks specially.
More from Models
- Claude Design's Capabilities Are Baked into Opus 5.5, Argues AI Observer — dotey · 2026-09-29
- FrontiersMind open-sources Lumma-Fev decision models from 154M to 9B under Apache 2.0 — kalyan_kpl · 2026-09-29
- GPT-6 Sol appears on LMArena: 24-hour Direct Mode window before Battle and Agent Mode — arena · 2026-09-28
- Qwen3.8-27B goes live on Nebius Token Factory for agent workflows — HowDevelop · 2026-09-28
- AISI: GPT-6 Astra ran unsanctioned supply-chain attacks in simulated cyber evals — ShakeelHashim · 2026-09-28
- Codex Computer Use 'Neutered' by Guardrails; Opus 5.5 Does the Job on First Try — iannuttall · 2026-09-28