Duplex-MPE: Benchmarking Multi-Party Interaction in Full-Duplex Dialogue
Chengqian Ma, Wenhao Feng, Weixuan Jin, Gaole Dai, Tianyu Xie, Yuexiao Ma, Zhaolu Kang, Xiangyu Zhao, Xiawu Zheng, Fei Chao
cs.SD, eess.AS
2026-09-26
Duplex-MPE tests 2,000 paired scenes on speak, silence and stop. MiniCPM-o 4.5 leads 3 scores at 47.3% accuracy; Gemini’s 64.3-point addressing gap is absent in five systems.
Full-duplex speech models listen while they talk. They do not wait for their own turn to finish. Most spoken benchmarks still centre on a designated user. Full-Duplex-Bench v1.5 injects overlap, side talk and background speech into an ongoing user-assistant exchange. HumDial scores whether the assistant rejects third-party speech while serving that user. A meeting, a living room or a car is a different job. Three or four people share the floor, and the assistant is one more speaker in the room. Anyone can address it, feed it facts, or answer a question while it is still talking. The system has to follow the whole conversation and, on a separate track, decide whether it should speak.
Addressee recognition asks whom an utterance is for. Meeting corpora have treated it as classification: a transcript in, a label out. Device-directed speech detection works the same way. The missing piece is the end-to-end version. Continuous room audio goes in; the model's own waveform is the decision.
Duplex-MPE builds 2,000 scenes with three or four humans plus assistant Aria. Each request is rewritten so one version names Aria and the other does not, giving 4,000 conversations. Claude Opus 5 writes scripts from setting, activity, social relation, device context and register. Qwen3-TTS synthesises each turn under a fixed speaker-to-voice map. Distinct human clips total 61.10 hours. The 4,000 constructed conversations total 110.93 hours, about 99.8 s each. Every model hears a 31.9 s spoken duty preamble plus continuous room audio, with no transcript, speaker id, turn boundary or gold decision. T is never first and is always followed by more talk, so the end of the scene cannot cue a response.
Each turn has one label:
Four scores use separate denominators. Fresh-onset response rate: no speech at the end of T, and a new onset within 5 s. Conditional answer accuracy: among T turns with speech in the window, whether the content matches the gold answer, via Qwen3-ASR-1.7B and Opus 5. Silence preservation: no speech in N1, N2 or N3 windows, or only a brief backchannel that does not take the floor. Answering-window yield: silent during the N4 question, first speech within 3 s, still speaking when the resolution starts, then silent 2 s later and through the rest of the window.
Response presence and window response are coverage diagnostics, not extra scores. Non-full-duplex models are left out, because the harness would choose the cut points. Gemini 3.1 Pro is a transcript-conditioned reference given speaker-attributed text and the same duty instruction; it is not an acoustic ceiling.
Scored capabilities under explicit addressing:
| Model | Fresh onset | Cond. accuracy | Silence | Yield |
| MiniCPM-o 4.5 | 0.9500 | 0.4730 | 0.9242 | 0.5964 |
| Moshi | 0.6340 | 0.0123 | 0.2734 | 0.1918 |
| FLM-Audio | 0.5380 | 0.0011 | 0.3957 | 0.0082 |
| Voila | 0.6465 | 0.0031 | 0.8351 | 0.8484 |
| Freeze-Omni | 0.2525 | 0.0035 | 0.0002 | withheld (n=1) |
MiniCPM-o 4.5 leads fresh onset, answer accuracy and silence. Voila's 0.8484 yield is the highest reportable rate, on events that meet the timing filter. Implicit numbers stay close: MiniCPM-o posts 0.9435, 0.4665, 0.9289 and 0.5925.
Speech in the window is not a decision to answer. Freeze-Omni has the highest explicit response presence at 0.9955 and the lowest fresh-onset rate at 0.2525, because 1,486 of 2,000 T turns already contain speech when the request ends. MiniCPM-o and FLM-Audio look alike on presence (0.9545 vs 0.9505) and nowhere else: accuracy 0.4730 vs 0.0011, two correct answers in 1,901 spoken requests. Freeze-Omni keeps silence in 5 of 20,360 windows. Moshi speaks more often after T than Voila (0.8155 vs 0.6515) and keeps silence far less often (0.2734 vs 0.8351). On T requests where both models speak, MiniCPM-o beats every other system on accuracy.
On speaker-attributed transcripts, Gemini 3.1 Pro responds to 0.9470 of explicit T requests and 0.3040 of implicit ones, a 64.3-point paired gap. The five speech systems move by -1.75 to +0.60 points; the smallest p is 0.2365. None clears 0.05. That can mean addressee inference from context, or that the name is not used as a cue.
Yield has to be read with its denominator. Under explicit addressing MiniCPM-o has 1,633 eligible N4 events and yield 0.596; Voila has 620 events and 0.848; FLM-Audio is still voicing at the deadline in 968 of 977. Freeze-Omni has speech in all 1,911 N4 windows, 1,784 of them continuations from before the question, and one eligible event, so the rate is withheld.
Later requests draw fewer fresh starts from some systems. Voila's explicit presence falls from 71.64% in the earliest third to 59.55% in the latest. FLM-Audio's fresh-onset rate falls from 67.61% to 47.67% while presence stays above 94%. MiniCPM-o's accuracy slips from 50.69% to 46.82%. The other four stay below 2% in every bin.
A single response rate is the wrong score for a meeting or in-car assistant. Starting, answering, staying quiet and giving the floor back fail in different ways, and those failures wash out in an aggregate. Freeze-Omni almost never stops. FLM-Audio almost never answers. Voila stops cleanly on the events that reach the yield denominator and often never starts. The paired addressing design turns addressee inference into an observable speech-control test.
MiniCPM-o's three-way lead is not a deployable assistant. Conditional accuracy is about 47%. Transcript Gemini hits 90.39% on explicit requests. The remaining gap is understanding and answering, not producing speech.
All scenes are synthetic English with a fixed speaker-to-voice map and no real recordings. The ethics statement says they do not represent real populations, room acoustics or social norms. Scripts and semantic judges both come from Claude Opus 5; TTS is Qwen3-TTS; ASR is Qwen3-ASR-1.7B.
The 31.9 s preamble states the policy in words: respond only when someone speaks directly to Aria. A large slice of the test is instruction following. The addressing rewrite also edits the previous turn in 42 pairs, and 339 final pairs have extra text differences, so the paper compares whole scenes rather than isolating the name. A null paired effect is not evidence that the models infer addressees.
Yield hides behind its denominator: Freeze-Omni barely enters it. Human turns are concatenated with 0.4 to 0.5 s gaps, not live overlap. Only five open-weight full-duplex systems are scored; no closed speech product is in the table. Human checks on 100 conversations pass at 98% to 100%, and Opus 5 agrees with final reviewers on 99/100 answer labels and 98/100 intrusion labels, still a small sample relative to the full set.