DeepInfra Serves 700B+ Tokens/Day on NVIDIA Blackwell Ultra B300s
gharik · x · 2026-08-06
DeepInfra announced it is now serving over 700 billion tokens per day just on OpenRouter, alongside significant volume via private endpoints.
Infrastructure-wise, it is fully deployed on NVIDIA Blackwell Ultra B300 chips within the US. The company also noted that open-source models like DeepSeek V4 Flash are now production-grade, achieving a 90% cache hit rate and drastically reducing the cost per token, signaling an inflection point in token economics.
More from Infra
- Proxmox VE Officially Supports ARM64, Enabling Mixed Architecture Clusters — unixterminal · 2026-08-06
- Developers Mock Google's Squandered GPU Ecosystem Advantage and TPU Hype — teortaxesTex · 2026-08-06
- Cloudflare Launches Gadgets: Secure Architecture for AI Codegen — AI Engineer · 2026-08-06
- Hippius on Bittensor: Decentralized S3 Storage with ~900TB Capacity — markjeffrey · 2026-08-06
- Qdrant 1.19 Released: Introduces TurboQuant & Memory Tiers — qdrant_engine · 2026-08-06
- Marvell Photonic Fabric Wins AI Infrastructure Award for Scaling Inference — BenBajarin · 2026-08-06