WavLate aligns words to audio moments with MaxSim, keeping 32 vectors per 10s clip

IgorCarron · x · 2026-11-21

Aarush introduced WavLate, built on mLateOn as the text encoder and MGA-CLAP's audio encoder. It keeps 32 vectors per 10 seconds of audio and one vector per word, then uses MaxSim to match each word to the moment in the clip that fits it best.

Original post →

More from Multimodal

Multimodal channel →