GPT-Live-1 tested on real phone calls: natural speech but serious instruction-following flaws
kolchinski · reddit · 2026-09-14
ThunderPhone (an AI phone-agent company) ran OpenAI's new full-duplex speech-to-speech model GPT-Live-1 on a real phone number with a 13k-token insurance qualification script, across a dozen real calls and 25 simulated ones.
The good: the most natural phone conversationalist they've deployed—one continuous audio stream, 1.3s to first audio, zero VAD or turn-taking logic needed; interruptions, backchannels, and mid-sentence corrections just work.
B2B problems:
- Literalism: prompts ignored—caller says "I own it now," model still reads the own/rent/parents menu; held across every prompt variant.
- "Yes" handled as "no" multiple times, re-asking or moving on as if declined.
- Thinks out loud to callers under pressure ("Hmm. Handling this one carefully...").
- Alphanumeric errors: added a letter when reading back a claim number (audio wrong, transcript right), swapped Q for X in Russian (both wrong).
- Non-English is fluent but heavily American-accented; one Luganda call was answered in Swahili.
The founder promises full transcripts and audio clips in the comments.
Related event: GPT-Live-1 Real-World Test: Most Natural Voice, Shaky Instruction-Following(2 posts)→
More from Models
- The impossible ticket: defining and rewarding away 'Claudeishness' in model style — menhguin · 2026-09-14
- Remember 2019? OpenAI withheld GPT-2 as 'too dangerous to release' — timigod · 2026-09-14
- ChatGPT validates random nonsense while Claude calls it meaningless, test shows — flowersslop · 2026-09-14
- Agent trace dataset hits 50k+ monthly downloads — author speculates on SFT and reward-hacking monitor uses — maksym_andr · 2026-09-14
- Author Finds Gemini 'Reliably Wrong' at Verifying Quote Sources, 0/2 in Tests — danbri · 2026-09-14
- Astrable: Open-Source Codex Plugin Pairs GPT-6 Astra with Claude Fable 5.1 — daniel_mac8 · 2026-09-14