Hugging Face ships caching and serving upgrades, TTFT 1.5x faster
osanseviero · x · 2026-11-18
Hugging Face's Omar Sanseviero announces caching and other serving improvements now live, making TTFT 1.5x faster as of this morning. The team is also working on token and tool efficiency plus harness improvements, aiming for more intelligence per second in future models.
Related event: Hugging Face Speeds Up Carbon Inference as Jeremy Howard Takes Interest(2 posts)→
More from Infra
- Google donates Agent Substrate to CNCF: 500ms restores, 10x density — krishnan · 2026-11-18
- Report: Nvidia brokered Alibaba-Megaspeed deal, Alice Huang ties run deeper than known — GaryMarcus · 2026-11-18
- n8n's open-source agent Nod: secure remote access via private reverse proxy — victorialslocum · 2026-11-18
- pci_eerie: Open-Source PCIe Soft Stack for Xilinx 7-Series FPGAs as a Drop-in PCIE_2_1 Replacement — IgorCarron · 2026-11-18
- Hugging Face Nears 1 Exabyte of Monthly Data Distribution as Open Model Downloads Explode — andimarafioti · 2026-11-18
- NVIDIA, AMD, Intel, Google and Meta join SGLang Consortium as v1.0 ships — heyshrutimishra · 2026-11-18