Fireworks launches Specialized Intelligence Index scoring models on domain benchmarks, cost and duration
sophiamyang · x · 2026-11-19
Fireworks AI has launched the Specialized Intelligence Index, a free leaderboard comparing open, closed, and specialized language models.
- Covers domain benchmark suites contributed by industry practitioners, executed per each benchmark's published methodology
- Each board retains native headline scores, per-task cost in USD (retries included), and wall-clock duration per task
- Also features Bradley-Terry-based Diagnosis Elo ratings centered at 1,000 within the evaluated pool
- Freely accessible, version dated 2026-11-18
Unlike general leaderboards, the index emphasizes specialized intelligence — cost-effective performance on specific domain tasks — as a complement for model selection.
More from Models
- GPT-6.2 Astra matches predecessor's MazeBench score at $225 vs $840 — patience_cave · 2026-11-20
- MazeBench: GPT-6.2 Sol narrows gap with Astra but still scores less at same price — patience_cave · 2026-11-20
- Gemini 4 Neon leads MazeBench on uncached token efficiency by a wide margin — patience_cave · 2026-11-20
- MazeBench: GPT-6.2 Astra tops at 16% while costing 25% of its predecessor — patience_cave · 2026-11-20
- AssemblyAI STT 3.7 Tops Voice Arena Streaming ASR Leaderboard at 2.45% WER — AssemblyAI · 2026-11-20
- Transparency Isn't Objectivity: Community Debates Bias in Model Reviews — RichardsonDx · 2026-11-19