Boson AI's Higgs Realtime Duplex: one 45B MoE that listens, watches, talks and renders its own face

smolix · x · 2026-11-06

Boson AI unveiled Higgs Realtime Duplex, a single 45B MoE model that simultaneously listens, watches, talks, renders its own face, and delegates hard problems to a larger backend model without pausing the conversation.

smolix's thread breaks down the architecture:

Streaming representation: all streams are cut into 160ms chunks — audio in 1 token/chunk, audio out 2, video in a 12-token frame every 6 chunks, face out 2 latents, text flows both ways, silence counts as a chunk. That's 2.6K tokens/min; 5 minutes of AV plus compacted history fits in 13-25K tokens.
Zero-shot voice & face: a short reference clip, one image, and a text prompt for behavior. Audio uses a causal flow-matching decoder; face uses a causal DiT conditioned on the reference image and last 2 latents for long-session stability.
Training: continued pretraining of a text LLM on 2T tokens of text, audio and video; post-training uses GRPO RL specialists for conversation, timing, delegation and task completion, distilled on-policy into one student, plus a self-improvement loop in noisy simulated chats.
Thinking & tools: thinking, tool calls and delegation are spans in the same token stream interleaved with listening/speaking; tool results arrive in stages and the model can start talking before they're complete; hard problems route to a much larger backend model with full shared context.

Original post →

Boson AI's Higgs Realtime Duplex: one 45B MoE that listens, watches, talks and renders its own face

More from Models