Transformers v5.21 ships FP8 KV-cache, MoE sharding, and new TTS models
LysandreJik · x · 2026-10-27
Hugging Face's Lysandre announces Transformers v5.21 (one release every 12 days since v5). New models include Pocket TTS (100M params, faster-than-real-time speech on CPU), Gradium Phonon TTS, MiniCPM-OCR, UltraVBert, Flint, and the Pocket TTS audio tokenizer.
The deeper work is at the infrastructure level:
- New FP8 KV-cache backend
- MoE experts now sharded across TP/EP ranks instead of replicated on every rank
- Input embeddings sharded by default under tensor parallelism
- Flash Attention works with static caches
- Chunked prefill resumes from existing cache; continuous batching keeps improving
These are primitives for inference engines and training stacks, not just features for model.generate() users.
More from Infra
- Thermal imaging suggests IREN data halls 4 and 2 commissioning, 3-4 halls billing by year-end — McDonaghMatthew · 2026-10-27
- Google and Meta launch TorchTPU, a native PyTorch backend for TPUs — emmanuelvivier · 2026-10-27
- WireGuard tutorial: private encrypted tunnels between devices on any network — victorialslocum · 2026-10-27
- DeepSeek-V4.1-Flash 1.75bpw quant released for 128GB Macs — bobby-chan · 2026-10-27
- Unsloth details running DeepSeek-V4.1-Flash locally: 1-bit GGUF fits 128GB RAM with 1M context — gnukeith · 2026-10-27
- SK hynix down 47% since July while DDR5 spot prices rose 31%: a striking disconnect — tengyanAI · 2026-10-27