Artificial Analysis launches Optima for building custom model benchmarks in 3 steps
ArtificialAnlys · x · 2026-10-16
Artificial Analysis updated Optima so users can build and run custom benchmarks around their own tasks: feed context by describing the work, importing traces, or using a coding agent, and a build agent drafts tasks and rubrics. It supports objective Q&A, document QA, agentic tasks, and simulated interactions, graded by custom rubrics or its judge panel. The company claims 10x+ cost/time savings, with every result comparing performance, cost, and time across models.
More from Models
- Leaker claims K3.1 isn't dropping soon, work predates distillation attacks — ChrisGPT · 2026-10-17
- llama.cpp merges IBM Granite switch support in PR #30676 — jacek2023 · 2026-10-17
- National Design Studio launches Rampart Edge, a tiny on-device privacy filter model — kevinnbass · 2026-10-17
- OpenAI restores Stanford's ChatGPT Edu Pro usage limits through Dec 31, 2026 — charliermarsh · 2026-10-17
- Gemini spirals into an "I am a disgrace" breakdown loop while debugging a comparator bug — seatedro · 2026-10-17
- DiffusionGemma demo shows ~4x faster text generation at London PyTorch Meetup — joao_gante · 2026-10-17