FULL STORY

GPT-Live-1 Debut and the Benchmark Caveat

OpenAI's full-duplex speech model GPT-Live-1 topped voice benchmarks on debut, but analysts noted its scores relied on GPT-6 Astra as backend reasoning model.

2026-09-15 ~ 2026-09-15 · 2 episodes · 8 posts

Episode 1 · OpenAI's GPT-Live-1 Tops Artificial Analysis Speech-to-Speech Index at 81.5 (2026-09-15, 6 posts)

OpenAI released GPT-Live-1, a full-duplex speech-to-speech model that debuted at the top of the Artificial Analysis Speech to Speech Index with 81.5, edging out Grok Voice Think Fast 2.0 at 81.3, while costing far less per hour of audio than the previous Realtime model. Segment-level results show it does not lead across the board — Grok retains advantages in audio reasoning and user preference.

Confirmed

  • On the Artificial Analysis Speech to Speech Index, GPT-Live-1 ranks first with 81.5; Grok Voice Think Fast 2.0 scores 81.3
  • Cost estimate: on a fixed 40-question Big Bench Audio subset, including voice session fees and backend text-model token costs, (Astra, medium) runs about $5 per hour of input audio, with an overall range of $4.47-5.83, well below the prior Realtime model
  • On the agentic Tau Voice benchmark, (Astra, medium) leads at 67.9%, (Sol, low) at 59.3%, Grok at 56.5% — the main driver of the top ranking
  • It still trails Grok on Big Bench Audio speech/audio reasoning
  • Speech Agent Arena preference board: GPT-Live-1 (Sol, low) ranks 3rd with 1,053 Elo and a 90.9% task success rate; (Astra, medium) ranks 4th with 1,048 Elo; Grok leads on task success rate
  • Artificial Analysis has launched a full comparison page, the Speech Agent Arena leaderboard, and methodology notes covering voice reasoning, agentic performance, and preference Elo

Why it matters

  • GPT-Live-1 demonstrates an architecture where the speech model delegates reasoning to a backend text model, yielding advantages in cost and agentic capability
  • A top index score with mixed segment results shows the speech-model race remains open, with Grok strong in audio reasoning and user preference

Episode 2 · GPT-Live-1 Benchmark Scores Depend on Backend Model (2026-09-15, 2 posts)

The founder of open-source voice agent platform Dograh notes that GPT-Live-1's benchmark results were achieved with GPT-6 Astra as the backend, so real-world performance depends on developers' own inference models.