Open-Sourcing NaFlexCLAP: Variable-Resolution Audio-Text Models
wightmanr · x · 2026-08-14
Prominent vision model researcher Ross Wightman has shared his latest experimental model collection, NaFlexCLAP, now open-sourced on Hugging Face.
- Architecture: The project combines the timm NaFlexViT audio encoder with a modern text encoder to build CLAP (Contrastive Language-Audio Pretraining) models supporting variable time and variable length inputs.
- Engineering: The author made extensive modifications to the OpenCLIP main branch to support variable resolution/aspect ratio encoders and matching webdataset pipelines, integrating existing audio CLAP weights.
- Availability: Multiple model versions (ranging from 160M to 240M parameters) are currently available and can be used directly via the OpenCLIP main branch.
Related event: OpenCLIP Overhaul Integrates Audio Models and Variable Resolutions(3 posts)→
More from Multimodal
- Kling 4.0 fully available, shows markedly better motion coherence in fast-action scenes — alifcoder · 2026-10-16
- MJev-Spatial: Single-Pass Classification Beats Autoregressive VLMs on Spatial Reasoning — mindlab-research · 2026-10-16
- H-DDCM: Hadamard Codebooks Make Diffusion-Based Image and Video Compression Scalable — Technion · 2026-10-16
- ComfyUI v0.41.0 ships with Flux3 image model and MiniMax-H3 LightVAE support — Fcking_Chuck · 2026-10-16
- AI-generated short film 'The Dragon Rider' glides cinematically above the clouds — jijo6686 · 2026-10-16
- Realistic candid Midjourney portrait prompt: natural skin, lived-in bedroom light — tisch_eins · 2026-10-16