Lychee-FD: Open-Source Full-Duplex Speech Model

机器之心 · wechat · 2026-07-16

The Lizhi team at Harbin Institute of Technology, Shenzhen, has released and open-sourced Lychee-FD, a native end-to-end full-duplex speech model. It aims to transition voice interactions from turn-based Q&A to human-like full-duplex communication characterized by continuous listening, real-time comprehension, and natural response.

The article identifies two root causes of the challenges in developing native full-duplex speech models: gradient conflicts between acoustic and semantic modeling within deep parameter spaces, and high-frequency speech signals diluting sparse text semantic supervision. This often forces a trade-off, making models either more fluent but cognitively degraded, or semantically accurate but with higher latency. To address this, the team proposed a hierarchical semantic-acoustic modeling framework: a shared shallow trunk branching into semantic, acoustic, and dialogue control channels at deeper levels, utilizing dense semantic alignment to mitigate dilution.

Experimentally, Lychee-FD achieved average improvements of 7.4% on SpokenQA and 28.5% on FullDuplexBench1.5, securing state-of-the-art performance across 10 metrics on 3 full-duplex voice interaction benchmarks. On the engineering side, the team adapted a real-time parallel multi-stream inference framework based on vLLM, parallelizing multiple channels to achieve a 2.96x speedup and a 23% reduction in VRAM usage. They also introduced a control head early exit strategy for faster interruption and response switching. The post features demos of digital humans and physical robots powered by the model, reiterating that it is fully open-source for easy reproduction and extension.

Original post →

More from Multimodal

Multimodal channel →