Pipecat open-sources Audience: a tiny classifier so voice agents know when they're being addressed

alexcovo_eth · x · 2026-11-18

Jon Taylor (Daily) demos an AI agent joining a multi-person meeting: each utterance first passes through a fast classifier that decides whether the agent is being addressed; if unsure it waits 2.5 seconds before responding. All thresholds are tunable.

Stack: orchestration via Pipecat, STT via Deepgram, transport via multi-track Daily. Good diarization plus a millisecond-scale classifier is the key.

Pipecat also open-sourced pipecat-ai/audience on Hugging Face (Apache 2.0, ONNX/Core ML): a small addressee-detection model for multi-character voice experiences. It reads a line, the speaker, and a card for each possible addressee, returning per-person likelihood, whether the line targets the whole group, and an honest "unclear" so a character can ask "Who, me?" Constraints: millisecond CPU inference. RC2.1 adds a 16-bit copy at half size with same answers.

Original post →

More from coding & agent

coding & agent channel →