Trelis releases first speech-to-speech pre-training dataset with dual token tracks and rich annotations

morgymcg · x · 2026-11-18

Trelis Research released s2s-pre-training-yodas3-mimi on Hugging Face, the first in a planned series of speech-to-speech datasets and models. It is a filtered, tokenized pre-training dataset for dual-track speech-to-speech models: long English speech spans (one or two speakers, little music) drawn from the Creative Commons YODAS3 YouTube dataset. It ships Mimi audio tokens instead of raw audio, with machine annotations including speaker-attributed word-timed transcripts (covering overlapping speech), vocal bursts like laughter, per-utterance emotion labels, and separate token tracks per speaker when two people are present.

Original post →

More from Multimodal

Multimodal channel →