Sori-1B: Audio-Grounded LM Trained From Scratch With No Text-Only Pretraining
Balance- · reddit · 2026-08-31
An SNU researcher released Sori-1B, a 1B-parameter audio-language model whose decoder is trained entirely from scratch on audio-paired text, with no text-only pretraining or pretrained-LM initialization.
Technical Highlights:
- Audio-Grounded: Designed to ground answers in audio rather than relying on text-only priors (experiments show AF3 retains 74% performance with silence, indicating text reliance).
- Architecture: Reuses NVIDIA's frozen Audio Flamingo Next encoder (61.5% of params); everything else (decoder, embeddings, custom audio-concept tokenizer) is trained from scratch.
- Training Scale: Trained on 7.4k hours / 4.75M examples using just 3x RTX 4090s.
- Capabilities: Supports MCQ, open QA, captioning, and ASR.
- License: Weights are gated for non-commercial/academic use only due to NVIDIA's encoder terms.
More from Multimodal
- Tokyo's Gemini AI Photo Booth turns visitors into plush toys and dioramas — bradneuberg · 2026-10-16
- Orchestrating a 100-image portrait of humanity for aliens with ChatGPT, Argon and p5.brush — dx8xb · 2026-10-16
- Indie Creator Builds an AI-Native Animated Show Solo — ComprehensiveLime654 · 2026-10-16
- Ideogram teases 4.5 arrival and open weights: 'Not yet, but soon' — East_Shoe8814 · 2026-10-16
- Opus 5.5 turns heads with surprisingly polished vertical video demo — FinanceYF5 · 2026-10-16
- 'Celestial Ocean Reverie' prompt template blends starry skies with dreamlike seascapes — LudovicCreator · 2026-10-16