Transformers' biggest audio overhaul: unified AudioProcessor architecture
LysandreJik · x · 2026-10-26
Hugging Face is rebuilding essentially the entire audio preprocessing stack in 🤗 Transformers, spearheaded by eustachelb.
- Architecture shift: moving away from per-model FeatureExtractor implementations toward a common AudioProcessor design where models describe their frontend while shared backends do the computation.
- Coverage: 30 audio frontends and 62 model mappings migrated, spanning Whisper, Wav2Vec2, Parakeet, SeamlessM4T, Qwen, Gemma, codecs, and TTS models.
- Explicit backends: backend="torch" for native PyTorch preprocessing (often very fast), backend="numpy" for portable NumPy — while preserving numerical compatibility with existing checkpoints.
- What's next: a foundation for new hardware backends, faster kernels, compilation, and optimizations that benefit many audio models at once.
The unified entrypoint is AutoAudioProcessor.frompretrained(...).
More from Research
- Study: AI reviewing AI misses most reward hacking; best models miss 25%+ of hacks — ronbodkin · 2026-10-26
- FLUX.3 Klein step-distilled to 4-8 steps with CFG baked in — Fit-Structure-6489 · 2026-10-26
- NLI alone rejects 5/6 new agent memories; a 0.45 similarity gate fixes it all — Alternative_Pin9598 · 2026-10-26
- AI folks, stop saying "biology" when you mean drug development — KevinKaichuang · 2026-10-26
- ostris pre-training "Primal VAE": 1-channel latents match modern 8x VAEs via patching — ostrisai · 2026-10-26
- Shanghai AI Lab-led team releases 100-hour egocentric wearable dataset for humanoid training — ChuckDBrooks · 2026-10-26