Kezek splits mono two-person conversations into per-speaker tracks for full-duplex dialogue training data
manifestro · hf · 2026-11-19
Manifestro released Kezek (paper, code, weights, demo) to address a data bottleneck for full-duplex spoken dialogue models: most available two-person conversational audio is a single mono mix, while full-duplex models need per-speaker tracks to learn turn-taking, overlaps, and backchannels.
- Two-stage approach: stage 1 uses a phase-insensitive, speaker-fingerprint-guided two-voice activity model that decides "who speaks when" and "who is who" once for the whole recording; stage 2 uses a conditioned time-domain separator to execute that decision, with outputs that always sum back to the input.
- Fidelity: original length and timing preserved; short replies ("yeah," "mm-hm") stay on the speaker's own track.
- New metrics: standard separation metrics miss dialogue-relevant failures, so Kezek reports short-reply retention (0.83 vs 0.69 for the baseline on Open Yan test), leakage, channel swaps, and coverage.
- Outperforms the open pyannote→DialogueSidon pipeline on the same recordings.
More from Multimodal
- HyperFlow: 14B distilled MiniMax H3 runs video gen on 6GB VRAM at 2x speed — cgpixel23 · 2026-11-19
- One Neon prompt turns a monstera plant into a walking, waving character — fofrAI · 2026-11-19
- LTX-2.3 long-form lessons: CFG=1 kills negative prompts, brightness drifts per clip — Chance-Thanks-1278 · 2026-11-19
- 6-minute Hubble documentary made with Claude and Jimeng CLI workflows — xiaohu · 2026-11-19
- Seedance 2.5 4K hands-on: AI video finally loses its plastic look — xiaohu · 2026-11-19
- Eerie AI-generated portraits: neon grimace faces and glowing-eyed shadows — azed_ai · 2026-11-19