Hugging Face ships caching and serving upgrades, TTFT 1.5x faster

osanseviero · x · 2026-11-18

Hugging Face's Omar Sanseviero announces caching and other serving improvements now live, making TTFT 1.5x faster as of this morning. The team is also working on token and tool efficiency plus harness improvements, aiming for more intelligence per second in future models.

Related event: Hugging Face Speeds Up Carbon Inference as Jeremy Howard Takes Interest(2 posts)→

Original post →

More from Infra

Infra channel →