Trelis releases first speech-to-speech pre-training dataset with dual token tracks and rich annotations
morgymcg · x · 2026-11-18
Trelis Research released s2s-pre-training-yodas3-mimi on Hugging Face, the first in a planned series of speech-to-speech datasets and models. It is a filtered, tokenized pre-training dataset for dual-track speech-to-speech models: long English speech spans (one or two speakers, little music) drawn from the Creative Commons YODAS3 YouTube dataset. It ships Mimi audio tokens instead of raw audio, with machine annotations including speaker-attributed word-timed transcripts (covering overlapping speech), vocal bursts like laughter, per-utterance emotion labels, and separate token tracks per speaker when two people are present.
More from Multimodal
- Dev makes short film with Fable 5.2 about the reality of building games with AI — TAbrodi · 2026-11-18
- World Labs Atlas demo lets you explore a sketch in first person — XRarchitect · 2026-11-18
- "They were always there": retro-film Batman-on-horse AI images go viral — gizakdag · 2026-11-18
- Karate Kid reimagined as a 1971 film with AI video generation — techhalla · 2026-11-18
- Retrolab: real-time film emulation node for ComfyUI with an Anti-AI slider — listopalafoto · 2026-11-18
- Kaigen 3D AI engine joins $100k game jam alongside Unreal Engine support — gdechichi · 2026-11-18