Marimba: frozen Qwen3-ASR features power a lightweight speech decision model that matches Qwen3-Omni on 6 of 9 tests
charcot-ai · hf · 2026-11-18
charcot.ai released Marimba, a typed-decision interface for speech built on frozen Qwen3-ASR 0.6B features with lightweight readouts, answering multiple questions (term spotting, turn-end detection, language ID, speaker change, PII detection) from a single encoding of each 8-second window.
- Lexical questions use zero-shot scoring of decoder hypotheses with Platt calibration; non-lexical ones use a question-conditioned head and a layer-18 turn-end readout.
- Results: decoder AUROC 0.80–0.92 on conversational sets, 0.92–0.97 on read speech; speaker-change detection AUROC 0.966–0.980, beating an ECAPA baseline and counting speakers better than pyannote 3.1; turn-end AUROC 0.930.
- Despite using 0.6B features, Marimba matches or beats Qwen3-Omni (30B-A3B) and Qwen2-Audio (7B) on 6 of 9 benchmarks, with calibrated probabilities suited to accept/defer decisions for voice agents.
- Code and weights are open-sourced (Charcot-AI/marimba-api, charcot-ai/marimba).
More from Multimodal
- Dev makes short film with Fable 5.2 about the reality of building games with AI — TAbrodi · 2026-11-18
- World Labs Atlas demo lets you explore a sketch in first person — XRarchitect · 2026-11-18
- "They were always there": retro-film Batman-on-horse AI images go viral — gizakdag · 2026-11-18
- Karate Kid reimagined as a 1971 film with AI video generation — techhalla · 2026-11-18
- Retrolab: real-time film emulation node for ComfyUI with an Anti-AI slider — listopalafoto · 2026-11-18
- Kaigen 3D AI engine joins $100k game jam alongside Unreal Engine support — gdechichi · 2026-11-18