New audio encoders called a huge leap: embeddings far more informative, audio-CoCa teased

wightmanr · x · 2026-11-17

Developer wightmanr tested the new medium and large audio encoder models and called them a "HUGE step forward" — the embeddings are far more informative than previous encoders.

He also floated an idea: a caption-generator approach in the OpenVision2 or CoCa style could be really interesting for audio. The OpenCLIP community is already eyeing a MaMMUT or CoCa + NaFlex audio-encoder mashup.

Related event: New Audio Embedding Encoders Tested: A Major Leap Forward(2 posts)→

Original post →

More from Multimodal

Multimodal channel →